Building model with capture of as built features and experiential data

ABSTRACT

An augmented virtual model of a commercial structure, such as a processing plant and/or a manufacturing plant which includes a virtual design, as built data and experiential measurement data. A User may view the augmented model in a virtual reality setting based with the content presented to the user based upon a location of the user at the time of viewing. Aspects of the Augmented Virtual Model are updated over time and utilized to track performance.

CROSS REFERENCE TO RELATED APPLICATIONS

The present application claims priority to Non Provisional patentapplication Ser. No. 16/161,823, filed Oct. 16, 2018 and entitledBUILDING MODEL WITH CAPTURE OF AS BUILT FEATURES AND EXPERIENTIAL DATAas a Continuation in Part Application; to Non Provisional patentapplication Ser. No. 15/887,637, filed Feb. 2, 2018 and entitledBUILDING MODEL WITH CAPTURE OF AS BUILT FEATURES AND EXPERIENTIAL DATAas a Continuation in Part Application: to Non Provisional patentapplication Ser. No. 15/703,310, filed Sep. 13, 2017 and entitledBUILDING MODEL WITH VIRTUAL CAPTURE OF AS BUILT FEATURES AND OBJECTIVEPERFORMANCE TRACKING as a Continuation in Part Application: and to NonProvisional patent application Ser. No. 15/716,133, filed Sep. 29, 2017and entitled BUILDING MODEL WITH VIRTUAL CAPTURE OF AS BUILT FEATURESAND OBJECTIVE PERFORMANCE TRACKING as a Continuation in PartApplication: and to Provisional Patent Application Ser. No. 62/462,347,filed Feb. 22, 2017 and entitled VIRTUAL DESIGN, MODELLING ANDOPERATIONAL MONITORING SYSTEM: and to Provisional Patent ApplicationSer. No. 62/531,955, filed Jul. 13, 2017 and entitled BUILDING MODELINGWITH VIRTUAL CAPTURE OF AS BUILT FEATURES; and to Provisional PatentApplication Ser. No. 62/531,975 filed Jul. 13, 2017 and entitledBUILDING MAINTENANCE AND UPDATES WITH VIRTUAL CAPTURE OF AS BUILTFEATURES; the contents of each of which are relied upon and incorporatedherein by reference.

FIELD OF THE INVENTION

The present invention relates to methods and apparatus for creatingmodels including virtual design and operation of a facility andcapturing actual build details and performance of a facility modeled.More specifically, the present invention presents methods and apparatusfor generating an immersive experience in a virtual facility that iscapable of emulating a physical facility. As described herein, afacility may include one or more of: a commercial building, a processingplant and a manufacturing plant.

BACKGROUND OF THE INVENTION

Traditional methods of using automated design tools, such as AutoDesk™,are focused on the generation of a design plan for use in constructionof a facility, such as a processing plant. An automated design tool maybe advantageous in the specifying of building aspects, materials andplacement of features. Aspects may include building features, such aswalls, ingress/egress, utilities and even equipment.

More sophisticated productions include three dimensional models viewablefrom one or more user selected vantage points within the model. However,in general, the automated modeling ends with the construction of thefacility that has been modeled. It may be desirable to have ability tooperate a design system that provides virtual modelling of facilityaspects from the site level all the way down to the operational aspectsof work material flow within equipment in the facility and therebetween. It may be further desirable where such a virtual modellingsystem may be supportive of operational monitoring systems on physicallyoperating facilities where virtual modelling may be compared to actualresults in a displayed format.

Such traditional methods of design of a Processing Facility areprimarily accomplished based upon a predefined objective. For example, afactory seeking a certain level of performance, such as a productionrate, may be designed with an appropriate layout and include machinerywith sufficient size and speed to meet the level of production sought.Design models are not able to quantify whether an intended level ofperformance has been met because they are not equipped with a means tocapture empirical data on an ongoing basis.

Similarly, while traditional methods of using automated design tools,such as AutoDesk™, have greatly increased the capabilities of virtualmodels of facilities, very little has been done to quantify a deployedperformance of design features, such as equipment layout, capacity,throughout consumables walls, ingress/egress, windows, ceiling designs,textures, building materials, placement of structural beams, utilities,machinery location, machinery type, machinery capacity equipment.

More sophisticated design systems include “virtual reality” models.Virtual reality models may include two dimensional and/or threedimensional views from one or more user selected Vantage Points withinthe model of the structure. However, in general, access to the automatedmodeling by support personnel ends upon the construction of the buildingthat has been modeled. Hard copy prints of a commercial building may beavailable; however there is no guarantee that a building was constructedaccording to design plans or which equipment and machinery will beencountered.

SUMMARY OF THE INVENTION

Accordingly, the present invention combines methods and apparatus intosystems that extend the usefulness of a model of a facility through thelife of the facility constructed and relates the model with real timeand historical data. The present invention draws upon designs,materials, equipment deployed, environmental conditions experienced andother tangible conditions to provide a user with one or more factorsincluding: total cost of ownership of a facility over the life of thefacility or a specified span of years, total throughput, Quality ofGoods Processed, Total Yield, Total human resources required to run thefacility, and other factors that are vital to successful operation of afacility. The present invention additionally provides that the factorslisted above may be calculated in a model and validated with real timeand/or historical data.

The present invention provides for automated apparatus for improvedmodeling of construction, Deployment and updating of a ProcessingFacility. The improved modeling is based upon generation of As Built andExperiential Data captured with one or both of Smart Devices and Sensorslocated in or proximate to the Processing Facility. The automatedapparatus is also operative to model compliance with one or moreperformance levels for the Processing Facility related to processing ofa Commercial Product.

User input Performance Metrics may include an allocation of an amount ofconsumables required for a level of Performance a Processing Facilitywill experience during Deployment of the Processing Facility. Design,repair, maintenance and upgrades to a Processing Facility are modeledwith the automated apparatus by incorporating “As Built” data that isdescriptive of a physical structure and “experiential” data derived fromexperiential sensor readings of sensors located within or proximate tothe physical structure. Sensors may be included as part of a ProcessingFacility Structure and/or transported by a User. The data isincorporated into a virtual model of the structure and used to determinePerformance Level.

In another aspect of the present invention, a virtual model of aProcessing Facility extends beyond a design stage of the structure intoan ‘As Built” stage of the structure and additionally includesgeneration and analysis of Experiential Data capturing conditionsrealized by the Processing Facility during a Deployment stage of thestructure.

In general, As Built and Experiential Data generated according to thepresent invention includes one or more of: image data, measurements,component specifications of placement; solid state; electrical; andelectromechanical devices (or combination thereof); generate datacapturing conditions experienced by a structure. In addition, a user mayenter data, such as for example, data descriptive of an action taken bya service technician into the AVM. As Built and Experiential Data may beaggregated for a single structure or multiple structures. A Likewise, aProcessing Facility may comprise a single structure or multiplestructures.

In one aspect, an AVM virtual model may receive as input one or morestated Performance Levels for which the Processing Facility is deployedto meet. As Built data and Experiential Data are generated and analyzedfor predicting a rate of success of achieving a stated Performance Leveland quantifying a rate of success of the stated Performance Level.

As Built data is collected that quantifies details of how a specificphysical structure was actually constructed. According to the presentinvention, a Processing Facility is designed and modeled in a 3D virtualsetting. As Built data is combined with a design model in a virtualsetting to generate an AVM. As Built data may reflect one or more of:fabrication of the Processing Facility; repair; maintenance; upgrades;improvements; and work order execution associated with the ProcessingFacility.

In addition, Experiential Data may be generated and entered into the AVMvirtual model of the structure. Experiential Data may include dataindicative of a factor that may be tracked and/or measured in relationto the Processing Facility. Experiential data is typically generated bySensors in or proximate to the Processing Facility and may include, byway of non-limiting example, one or more of: vibration Sensors (such asaccelerometers and piezo electro devices); force transducers;temperature sensing devices; amp meters, ohmmeters, switches, motiondetectors; light wavelength capture (such as infrared temperatureprofile devices), water flow meters; air flow meters; and the like. Someexamples of Experiential Data may include: details of operation ofequipment or machinery in the Processing Facility; vibrationmeasurements; electrical current draws; machine run times, machine runrates, machine run parameters; interior and/or exterior temperatures;opening and closings of doors and windows; weight loads; preventivemaintenance; cleaning cycles; air circulation; mold contents; thermalprofiles and the like. Automated apparatus captures empirical dataduring construction of the Processing Facility and during Deployment ofthe Processing Facility.

In some embodiments, empirical data may be used to track achievement ofa defined Performance Level. Empirical data may also be used scheduleDeployment of assets for the Processing Facility. Deployment of assetsmay include one or more of: human resources in the form of man hours;equipment, consumer devices, fixtures, machinery, maintenance items andthe like. Deployment of assets may also include tracking of pecuniaryequivalents of assets or other fungible equivalent.

In preferred embodiments, dedication of one or more assets for which avalue may be associated, is translated into a modification of theProcessing Facility. A return on investment for a modification may becaptured via the automated apparatus as additional empirical data.Empirical data may also be used to increase the accuracy of virtualmodels generated by the automated apparatus that are descriptive of theProcessing Facility. A return on investment (“ROI”) may be measuredaccording to variables involved, for example, increased energyefficiency resulting from preventative maintenance procedures may bemeasured according to an amount of power required for similarPerformance pre and post the PM procedure, or fewer user complaints. Inanother aspect, return on investment may be translated to a fungibleitem, such as currency or other financial amount, in order to best trackdisparate variables on a comparative basis.

By way of additional example, it may be determined that waterconsumption in a particular Processing Facility, or a particular classof processing plants, will be analyzed to determine if it is prudent tomake modifications to the particular Processing Facility or class ofprocessing plants. The automated apparatus of the present invention willinclude As Built data for features of a structure that is accessed whilemodeling proposed modifications and upgrades. Relevant As Built Featuresmay include features for which relevancy may seem obvious, such as, forexample, one or more of: utility requirements, electrical, chemicalsupply, chemical waste disposal, air handling equipment, hoods, exhaustand filtering; plumbing; machinery models and efficiency. In addition,other As Built Features, for which relevancy may not seem obvious, butwhich unstructured queries draw a correlation may also be included.Location of machinery relative to other machinery may also be deemedrelevant by unstructured query analysis. An unstructured query ofcaptured data quantifying actual chemical, atmosphere and water usagemay determine that certain configurations better meet an objective thanothers. It may later be determined that the single story structure ismore likely to have a consistent internal temperature, lighting, ambientparticulate or other trends.

As discussed more fully below, captured data may include empiricalquantifications of a number of times a piece of machinery cycles on andoff, vibrations within a structure, temperature within a structure,doors opening and closing, quantity of products processed, hours ofoccupancy of the structure and other variable values. Captured data mayalso be used to generate a determination of how a structure is beingused, such as production cycles, quality, yield, rates, volumes, etc. Asdiscussed more fully below, empirical Sensor data associated with howparticular personnel behaves within a Processing Facility may also becorrelated with structure Performance based upon who occupies aparticular structure, when they occupy and for how long.

The automated apparatus combines a model of a structure that has beendesigned and provides precise additions to the model based upon datacapture of features actually built into the structure. Service callsthat may include one or more of: repairs, upgrades, modifications andadditions (hereinafter generally referred to as “Service Call”), mayaccess data indicating an AVM combined with precise features included ina building represented by As Built data, as well as Experiential Dataand technical support for the features, maintenance logs and schedules,“how to” documentation and video support, virtual connection tospecialists and experts, and a time line of original As Built detailsand subsequent modifications. Modifications may include repairs, updatesand/or additions to a structure.

The improved methods taught herein provide for the performance ofrepairs, maintenance and upgrades via access to a system thatincorporates “As Built” data into the AVM. Geolocation and directionwill be used to access virtual reality representations of a structureincluding actual “As Built Imagery” incorporated into the AVM thataccurately indicates locations and types of features and also providesimages or other captured data. Exemplary data may include As Builtlocations of structural components (beams, headers, doorways, windows,rafters etc.); HVAC, electrical, plumbing, machinery, equipment, etc.Virtual repair may include “how to” instructions and video, technicalpublications, aggregated of similar repair orders and the like. Anonsite technician may verify correct location of an equipment unit basedupon GPS, triangulation, direction determinations.

A virtual reality model may additionally include virtual operation ofequipment and use of modeled structure based upon aggregated data fromone or more As Built structures. Upon conclusion of a repair,maintenance, upgrade or addition. Additional information quantifyingtime, place, nature of procedure, parts installed, equipment, newcomponent location etc. may be captured and incorporated into a virtualmodel.

Some embodiments of the present invention include capturing data ofprocedures conducted during preventive maintenance and/or a Service Calland inclusion of relevant data into a virtual model. Precise datacapture during a Service Call or during construction may include actuallocations of building features such as, electrical wiring andcomponents, plumbing, joists, headers, beams and other structuralcomponents. Data capture may be ongoing over time as the building isused and modified, or updated during the life of a structure (sometimesreferred to herein as the “Operational” or “Deployed” stage of thestructure).

An Operational Stage may include, for example: occupation and use of aCommercial Property, as well as subsequent modifications, repairs andstructure improvements. The Commercial Property may include one or moremodeled structures, such as: a factory, processing plant, fabricationfacility, server farm, power generator facility, an outbuilding andfacilities included in a Commercial Property. Smart Devices with uniquemethods of determining a location and direction of data capture areutilized to gather data during construction of modeled buildings orother structures and during Deployment of the structures during theOperational Stage.

In general, Smart Devices provide ongoing collection of “As Built” and“Deployed” data that is captured during construction and Deployment of acommercial building. The collected data is further correlated withdesign data and used to track Performance of features included in adesign of process plants and/or features included within the confines ofa Commercial Property parcel (“Commercial Property”).

In another aspect, collected data may be used to predict Performance ofa Commercial Property based upon features built into the structure andconditions experienced by the Commercial Property. As Built data mayinclude modifications to a Commercial Property that are made during aconstruction phase, and/or during a Deployment phase, of a CommercialProperty life cycle. Similarly, as Deployed data may include detailsquantifying one or more of: machine operators, production quantity,yield, quality level, usage, maintenance, repairs and improvementsperformed on the Commercial Property.

In still another aspect of the present invention, predictive analyticsmay be performed to predict a life of various components included in theCommercial Property. Maintenance procedures and replacement ofconsumables or other parts may also be budgeted and scheduled based upona correlation of a) design data; b) As Built data; and c) as used data.In addition, contemplated improvements may be modeled according to anexpected return on investment (“ROI”). An expected ROI may be calculatedaccording to one or more of: an objective level of measurements, anamount of a fungible item, such as kilowatt, gallon, currency, volume orother quantity expended during the life of Deployment; and satisfactionof users and Performance.

Predictive analytics may include monitoring use of equipment andmachinery. The monitoring may include data collection that is stored ina controller and analyzed, such as, via artificial intelligenceroutines. In some embodiments, data gathered during monitoring may betransmitted to a centralized location and aggregated with other similartype buildings, equipment and machinery. Analytic profiles may begenerated. Predicted Performance and failures may be generated and usedto schedule Service Calls before a physical failure occurs. Profiles mayinclude degrees of usage, consumables, electric current draws,vibration, noise, image capture and the like.

In some embodiments, production rates, yields, cost of build, and costof Deployment, including maintenance costs incurred during Deployment ofa Commercial Property may be calculated and included into one or moreof: a production value of a Commercial Property including a ProcessingFacility; a sale price of a Commercial Property; and a lease value of aCommercial Property and overall asset volume of the Commercial Property.

In still another aspect, a comprehensive cost of build and Deploymentmay be amortized over a term of years. Still further, an amortized costmay be included in a scheduled payment for a term of years, such as, forexample a monthly mortgage payment, wherein the monthly mortgage paymentincludes Total Cost of Ownership. Total Cost of Ownership may includeone or more of acquisition, deployment, repair and maintenance andenergy usage. In some respects, a sale price that includes Total Cost ofOwnership may have favorable tax implications for one of or both Buyerand Seller.

Still another aspect includes generation of virtual reality userinterfaces accessing the AVM based upon a) design data; b) As Builtdata; c) as used data; and d) improvement data. A virtual reality userinterface may be accessed as part of one or more of: a maintenanceroutine; to support a change order for the Commercial Property; and tocontemplate improvements in a Commercial Property. As Built and asdeployed data may include data quantifying repairs and updates to theCommercial Property.

In some embodiments, a) design data; b) As Built data; c) ExperientialData; and d) Lead Actions and Lag Benefit measurements, as they relateto multiple Commercial Properties may be aggregated and accessed tosupport one or more Commercial Properties. Access to aggregated data mayinclude execution of artificial intelligence (AI) routines. AI routinesmay include, by way of non-limiting example; structured algorithms andunstructured queries operative to predict Performance metrics andmaintenance needs. AI routines may access both initial designs and dataaggregated during build and deployment stages of the CommercialProperty.

The details of one or more examples of the invention are set forth inthe accompanying drawings and the description below. The accompanyingdrawings that are incorporated in and constitute a part of thisspecification illustrate several examples of the invention and, togetherwith the description, serve to explain the principles of the invention:other features, objects, and advantages of the invention will beapparent from the description, drawings, and claims herein.

DESCRIPTION OF THE DRAWINGS

The accompanying drawings, that are incorporated in and constitute apart of this specification, illustrate several embodiments of theinvention and, together with the description, serve to explain theprinciples of the invention:

FIG. 1A illustrates a block diagram of inter-relating functions includedin automated systems according to the present invention.

FIG. 1B illustrates geolocation aspects that may be used to identify aCommercial Property and corresponding data and predictions.

FIG. 1C illustrates a block diagram of ongoing data capture via SmartDevices and Sensors and support for predictive modeling based upon thesmart data capture.

FIG. 1D illustrates an exemplary Progressive Facility layout withvarious equipment delineated in a top-down representation according tosome embodiments of the present invention.

FIG. 1E illustrates a diagram of a user and directional image data.

FIG. 2 illustrates a block diagram of an Augmented Virtual Modelingsystem.

FIGS. 3A-3F, are illustrations of exemplary aspects of collecting anddisplaying data of a Processing Facility generated during constructionof the Processing Facility.

FIG. 3G illustrates an exemplary key component of the model system, witha Performance monitor providing data via a communication system to themodel system.

FIG. 3H illustrates an exemplary virtual reality display in concert withthe present invention.

FIGS. 4A, 4B, and 4C illustrate an exemplary method flow diagrams withsteps relating to processes.

FIG. 5 illustrates location and positioning devices associated within aProcessing Facility.

FIG. 6 illustrates apparatus that may be used to implement aspects ofthe present invention including executable software.

FIG. 7 illustrates an exemplary handheld device that may be used toimplement aspects of the present invention including executablesoftware.

FIG. 8 illustrates method steps that may be implemented according tosome aspects of the present invention.

FIGS. 9A-D illustrates views of an AVM via a wearable eye displayaccording to some aspects of the present invention.

FIGS. 10A-C illustrates viewing areas of an AVM according to someaspects of the present invention.

FIGS. 11A-C illustrates vertical changes in an AVM viewable areaaccording to some aspects of the present invention.

FIG. 12 illustrates designation of a direction according to some aspectsof the present invention.

DETAILED DESCRIPTION

The present invention relates to methods and apparatus for improvedmodeling, Deployment and updating of commercial Processing Facilitybased upon As Built and Experiential Data. As Built and ExperientialData may quantify an allocation of resources required for a level ofProcessing Facility Performance during Deployment of the facility.Design, repair, maintenance and upgrades to a Processing Facility aremodeled with an automated system that incorporates “As Built” data and“Experiential” data into a virtual model of the structure to determine alevel of performance of the Processing Facility.

The present invention provides automated apparatus and methods forgenerating improved Augmented Virtual Models (sometimes referred toherein as an “AVM”) of a Processing Facility; the improved AVMs arecapable of calculating a likelihood of achieving stated PerformanceLevel specified by a user. In addition, the improved model may beoperative to generate target Performance Metrics based upon As Built andExperiential Data.

The Augmented Virtual Model of the Commercial Property may include aconceptual model and progress through one or more of: a) a design stage;b) a build stage; c) a Deployment stage; d) a service stage; e) anmodification stage; and f) a dispensing stage. As discussed more fullyherein, an AVM according to the present invention include originaldesign data matched to As Built data captured via highly accurategeolocation, direction and elevation determination. As Built data ismatched with a time and date of data acquisition and presented in twodimensional (2D) and three dimensional (3D) visual representations ofthe Commercial Property. The augmented models additionally include datarelating to features specified in a Commercial Property design and datacollected during building, Deployment, maintenance and modifications tothe Commercial Property. In some embodiments, a fourth dimension of timemay also be included.

An Augmented Virtual Model includes a three or four dimensional model ina virtual environment that exists parallel to physical embodimentsmodeled in the Augmented Virtual Model. Details of one or more physicalstructures and other features within a commercial real estate parcel aregenerated and quantified and represented in the Augmented Virtual Model.The Augmented Virtual Model exists in parallel to a physical structurein that the AVM includes virtual representations of physical structuresand additionally receives and aggregates data relevant to the structuresover time. The aggregation of data may be one or more of: a) accordingto an episode (i.e. onsite inspection, repair, improvement etc.); b)periodic; and c) in real time (without built in delay).

The experience of the physical structure is duplicated in the virtualAugmented Virtual Model. The Augmented Virtual Model may commence via anelectronic model generated via traditional CAD software or other designtype software. In addition, the AVM may be based upon values forvariables, including one or more of: usage of a structure; usage ofcomponents within the structure; environmental factors encounteredduring a build stage or Deployment stage; and metrics related toPerformance of the structure. The metrics may be determined, forexample, via measurements performed by Sensors located in and proximateto structures located on the Commercial Property.

In another aspect, an Augmented Virtual Model may be accessed inrelation to modelling achievement of a stated Performance Level.Accurate capture of As Built Features and aggregated data of similarbuildings, equipment types, machinery and usage profiles assist in oneor more of: predicting Performance Level, Yield, Quality, Volume ofProduction, selecting appropriate technicians to deploy to a servicecall; providing correct consumables and replacement parts, scheduling apreventative maintenance; scheduling building, equipment and/ormachinery upgrades; matching a building, equipment and machinerycombination of a particular type of Deployment; providing on siteguidance during the Service Call; providing documentation relevant tothe building, equipment and machinery; providing access to remoteexperts that guide onsite technicians.

In some embodiments, a technical library specific to a particularproperty and location within the property may be maintained for eachCommercial Property and made accessible to an onsite technician and/orremote expert. The library may include, but is not limited to:structure, equipment/machinery manuals; repair bulletins, andrepair/maintenance. Appropriate how to videos may also be made availablebased upon an AVM with As Built and Experiential Data.

In another aspect, a parts ordering function may be included in theAugmented Virtual Model. Augmented parts ordering may allow a technicianto view an ordered part and view a virtual demonstration of the part inuse and procedures for replacing the part.

Aspects of the Augmented Virtual Model may be presented via a userinterface that may display on a tablet or other flat screen, or in someembodiments be presented in a virtual reality environment, such as via avirtual reality headset.

The present invention additionally provides for an Augmented VirtualModel to forecast Future Performance of a Commercial Property based uponthe values of variables included in data aggregated during the design,build and Deployment of the Commercial Property sometimes referred toherein as: a) Design Features; b) As Built data; and c) as Deployeddata.

The improved modelling system incorporates “As Built” data into theimproved design model. Subsequently, an onsite or remote technician mayaccess the As Built data to facilitate. The As Built data is generatedand/or captured via highly accurate geolocation, direction and elevationdetermination. Based upon the geolocation, direction and elevationdetermination, As Built data is incorporated into a design model at aprecise location within the AVM. In some embodiments, a time and date ofdata acquisition may be associated with updates to aspects of theimproved AVM such that a chronology of changes exists within the AVM.

Original design aspects and updated design aspects may be presented intwo dimensional (2D) and three dimensional (3D) visual representationsof the Commercial Property. The present invention provides forsystematic updates to As Built data during a Deployment of theCommercial Property. Updated data may verify and/or correct previouslyincluded data and also be used to memorialize modifications made duringa Service Call or modification to a Commercial Property.

Some exemplary embodiments may include updates to an AVM that include,one or more of: quantifying a make and model of equipment and machineryon site; time and date notation of change in location specific data;Model accessed and/or updated according to XYZ and distance data; XYdata may include high level location designation within the streetaddress via triangulation (i.e. such as a street address) and highlyspecific position designation (i.e. particular room and wall);combination of two types of position data; GPS, Differential GPS;references used during triangulation; aggregate data across multiplestructures for reference; designs that perform well; designs that fail;popularity of various aspects; access to and/or generation of, multipleAugmented Virtual Models; original and modified model versions; indexaccording to date/time stamp; index according to feature; indexaccording to popularity; index according to cost; index according toUser specific query; plumbing; electrical; HVAC; chemical, raw material,structural; access areas (i.e. crawl spaces, attics); periodic data andposition capture with camera/Sensor attached to a fixed position; andduring one or more of: repair/maintenance/updates.

Accordingly, actual “As Built’ imagery and location data is incorporatedinto the design model to accurately indicate a location and type offeature included in a structure, and provide “pictures” or othercaptured data. Exemplary data may include As Built locations ofstructural components (beams, headers, doorways, windows, rafters etc.);HVAC, electrical, plumbing, machinery, equipment, etc. A virtual realitymodel may additionally include virtual operation of machinery andequipment and use of a Processing Facility based upon aggregated datafrom the structure, as well as annotations and technical specificationsrelating to features included in the As Built model of a ProcessingFacility identified by time, date, geolocation and direction.

In some embodiments, an initial digital model may be generated accordingto known practices in the industry. However, unlike previously knownpractices, the present invention associates an initial digital modelwith a unique identifier that is logically linked to a geolocation andone or both of date and time designation, and provides updates to theoriginal model based upon data captured at the geolocation during arecorded timeframe. In this manner, a Virtual Reality Simulation isgenerated that logically links a digital model to a specific geographiclocation and actual As Built data at the specific geographic location.The updated model may be virtually accessed from multiple locations suchas a field office, onsite, a technical expert, a financial institution,or other interested party.

In some preferred embodiments, the geographic location will be providedwith accurately placed location reference points. The location referencepoints may be accessed during activities involved in a Service Call onthe Commercial Property, such as a repair or upgrade to a structure orother structures included within a property parcel surrounding thestructure. Accuracy of the reference points may or may not be associatedwith location relevance beyond the Commercial Property, however they domaintain accuracy within the Commercial Property.

Preferred embodiments may also include reference points accuratelyplaced within a structure Processing Facility located on the CommercialProperty. As further discussed below, the reference points may include,by way of non-limiting example, a wireless transmission data transmitteroperative to transmit an identifier and location data; a visualidentifier, such as a hash code, bar code, color code or the like; aninfrared transmitter; a reflective surface, such as a mirror; or othermeans capable of providing a reference point to be utilized in atriangulation process that calculates a precise location within thestructure or other structure.

Highly accurate location position may be determined via automatedapparatus and multiple levels of increasingly accurate locationdetermination. A first level may include use of a GPS device providing areading to first identify a Commercial Property. A second level may useposition transmitters located within, or proximate to, the CommercialProperty to execute triangulation processes in view of on-site locationreferences. A GPS location may additionally be associated with a highlevel general description of a property, such as, one or more of: anaddress, a unit number, a lot number, a taxmap number, a countydesignation, Platte number or other designator. On-site locationreferences may include one or more of: near field radio communicationbeacons at known X-Y position reference points; line of sight withphysical reference markers; coded via ID such as Bar Code, Hash tag, andalphanumeric or other identifier. In some embodiments, triangulation maycalculate a position within a boundary created by the reference pointsto within millimeter range. In some embodiments, Differential GPS may beused to accurately determine a location of a Smart Device with a subcentimeter accuracy.

In addition to a position determination, such as latitude and longitude,or other Cartesian Coordinate (which may sometimes be indicated as an “Xand Y” coordinate) or GPS coordinate, the present invention provides fora direction (sometimes referred to herein as a “Z” direction andelevation) of a feature for which As Built data is captured and importedinto the AVM.

According to the present invention, a direction dimension may be basedupon a movement of a device. For example, a device with a controller andan accelerometer, such as mobile Smart Device, may include a userdisplay that allows a direction to be indicated by movement of thedevice from a determined location acting as a base position towards anAs Built feature in an extended position. In some implementations, theSmart Device may first determine a first position based upontriangulation with the reference points and a second position (extendedposition) also based upon triangulation with the reference points. Theprocess of determination of a position based upon triangulation with thereference points may be accomplished, for example via executablesoftware interacting with the controller in the Smart Device, such as,for example via running an app on the Smart Devices.

In combination with, or in place of directional movement of a deviceutilized to quantify a direction of interest to a user, some embodimentsmay include an electronic and/or magnetic directional indicator that maybe aligned by a user in a direction of interest. Alignment may include,for example, pointing a specified side of a device, or pointing an arrowor other symbol displayed upon a user interface on the device towards adirection of interest.

In a similar fashion, triangulation may be utilized to determine arelative elevation of the Smart Device as compared to a referenceelevation of the reference points.

It should be noted that although a Smart Device is generally operated bya human user, some embodiments of the present invention include acontroller, accelerometer, data storage medium, Image Capture Device,such as a Charge Coupled Device (“CCD”) capture device and/or aninfrared capture device being available in a handheld or unmannedvehicle.

An unmanned vehicle may include for example, an unmanned aerial vehicle(“UAV”) or ground level unit, such as a unit with wheels or tracks formobility and a radio control unit for communication.

In some embodiments, multiple unmanned vehicles may capture data in asynchronized fashion to add depth to the image capture and/or a threedimensional and 4 dimensional (over time) aspect to the captured data.In some implementations, UAV position will be contained within aperimeter and the perimeter will have multiple reference points to helpeach UAV (or other unmanned vehicle) determine a position in relation tostatic features of a building within which it is operating and also inrelation to other unmanned vehicles. Still other aspects includeunmanned vehicles that may not only capture data but also function toperform a task, such as paint a wall, drill a hole, cut along a definedpath, or other function. As stated throughout this disclosure, thecaptured data may be incorporated into the virtual model of a ProcessingFacility.

In another aspect, captured data may be compared to a library of storeddata using image recognition software to ascertain and/or affirm aspecific location, elevation and direction of an image capture locationand proper alignment with the virtual model. Still other aspects mayinclude the use of a compass incorporated into a Smart Device.

In still other implementations, a line of sight from a Smart Device,whether user operated or deployed in an unmanned vehicle, may be used toalign the Smart Device with physical reference markers and therebydetermine an XY position as well as a Z position. Electronic altitudemeasurement may also be used in place of, or to supplement, a knownaltitude of a nearby reference point. This may be particularly useful inthe case of availability of only a single reference point.

Reference points may be coded via identifiers, such as a UUID(Universally Unique Identifier), or other identification vehicle. Visualidentifiers may include a bar code, hash tag, Alphanumeric or othersymbol. Three dimensional markers may also be utilized.

By way of non-limiting example, on site data capture may includedesignation of an XYZ reference position and one or more of: imagecapture; infra-red capture; Temperature; Humidity; Airflow;Pressure/tension; Electromagnetic reading; Radiation reading; Soundreadings (i.e. level of noise, sound pattern to ascertain equipmentrunning and/or state of disrepair), and other vibration or Sensorreadings (such as an accelerometer or transducer).

In some embodiments, vibration data may be used to profile use of thebuilding and/or equipment and machinery associated with the building.For example, vibration detection may be used to determine a machineoperation, including automated determination between proper operation ofa piece of equipment and/or machinery and faulty operation of theequipment and/or machinery. Accelerometers may first quantify facilityoperations and production speed and/or capacity during operations.Accelerometers may also detect less than optimal performance ofequipment and/or machinery. In some embodiments. AI may be used toanalyze and predict proper operation and/or equipment/machinery failurebased upon input factors, including vibration patterns captured.Vibrations may include a “signature” based upon machine type andlocation within a structure human related activity, such as, by way ofnon-limiting example: machine and foot traffic, physical activities,machine operations, machine failure, raised voices, alarms and alerts,loud music, running, dancing and the like, as well as a number ofmachines and/or people in the building and a calculated weight andmobility of the people.

Vibration readings may also be used to quantify operation of machineryand equipment associated with the building, such as HVAC, circulatorsand water pumps. Vibration data may be analyzed to generate profiles forproperly running equipment and equipment that may be faulty and/orfailing. The improved virtual model of the present invention embodied asan AVM may be updated, either periodically or on one off occasions, suchas during a service call or update call.

In some embodiments, a fourth dimension in addition to an XYZ dimensionwill include date and time and allow for an historical view of a life ofa structure to be presented in the virtual model. Accordingly, in someembodiments, onsite cameras and/or Sensors may be deployed and data maybe gathered from the on-site cameras and Sensors either periodically orupon command. Data gathered may be incorporated into the improvedvirtual model.

In still another aspect, the AVM may aggregate data across multipleCommercial Properties and buildings. The aggregated data may includeconditions experienced by various buildings and mined or otherwiseanalyzed, such as via artificial intelligence and unstructured queries.Accordingly, the AVM may quantify reasons relating to one or more of:how to reposition machines, route workflow or otherwise improve, designsthat work well; designs that fail; popular aspects; generate multipleVirtual Models with various quantified features; original and modifiedmodel versions and almost any combination thereof.

Although data may be gathered in various disparate and/or related ways,an aggregate of data may be quickly and readily accessed via thecreation of indexes. Accordingly, indexes may be according to one ormore of: date/time stamp; feature; popularity; cost; User specificquery; Plumbing; Electrical; HVAC; Structural aspects; Access areas;Periodic data and position capture with camera/Sensor attached to afixed position; during construction; during modification; duringDeployment; airflow; HVAC; machinery; traffic flows during use ofstructure; audible measurements for noise levels; and almost any otheraspect of captured data.

In another aspect, an Augmented Virtual Model may receive datadescriptive of generally static information, such as, one or more of:product specifications, building material specifications, productmanuals, and maintenance documentation.

Generally static information may be utilized within the AugmentedVirtual Model to calculate Performance of various aspects of aCommercial Property. Dynamic data that is captured during one of: a)design data; b) build data; and c) deployed data, may be used to analyzeactual Performance of a Commercial Property and also used to update anAugmented Virtual Model and increase the accuracy of additionalpredictions generated by the Augmented Virtual Model. Maintenancerecords and supporting documentation may also be archived and accessedvia the AVM. A variety of Sensors may monitor conditions associated withone or both of the structure and the parcel. The Sensors and generateddata may be used to extrapolate Performance expectations of variouscomponents included in the Augmented Virtual Model. Sensor data may alsobe aggregated with Sensor data from multiple Augmented Virtual Modelmodels from multiple structures and/or Commercial Properties andanalyzed in order to track and/or predict Performance of a structure ormodel going forward.

Glossary

“Ambient Data” as used herein refers to data and data streams capturedin an environment proximate to a Vantage Point and/or an equipment itemthat are not audio data or video data. Examples of Ambient Data include,but are not limited to Sensor perception of: temperature, humidity,particulate, chemical presence, gas presence, light, electromagneticradiation, electrical power, moisture and mineral presence.

“Analog Sensor” and “Digital Sensor” as used herein include a Sensoroperative to quantify a state in the physical world in an analogrepresentation.

“As Built” as used herein refers to details of a physical structureassociated with a specific location within the physical structure orparcel and empirical data captured in relation to the specific location.

“As Built Features” as used herein refers to a feature in a virtualmodel or AVM that is based at least in part upon empirical data capturedat or proximate to a correlating physical location of the feature.Examples of As Built Features include placement of structural componentssuch as a wall, doorway, window, plumbing, electrical utility, machineryand/or improvements to a parcel, such as a well, septic, electric orwater utility line, easement, berm, pond, wet land, retaining wall,driveway, right of way and the like.

“As Built Imagery” (Image Data) as used herein shall mean image datagenerated based upon a physical aspect.

“Augmented Virtual Model” (sometimes referred to herein as “AVM”): asused herein is a digital representation of a real property parcelincluding one or more three dimensional representations of physicalstructures suitable for commercial use and As Built data captureddescriptive of the real property parcel. An Augmented Virtual Modelincludes As Built Features of the structure and may include improvementsand features contained within a Processing Facility.

“Commercial Property” as used herein shall mean one or more real estateparcels suitable for a deployed Processing Facility that may be modeledin an AVM.”

“Directional Indicator” as used herein shall mean a quantification of adirection generated via one or both of: analogue and digitalindications.

“Directional Image Data” as used herein refers to image data capturedfrom a Vantage Point with reference to a direction. Image data mayinclude video data.

“Directional Audio” as used herein refers to audio data captured from aVantage Point within or proximate to a Commercial Property and from adirection.

“Deployment” as used herein shall mean the placement of one or more of:a facility machinery and an equipment item into operation.

“Deployment Performance” as used herein shall mean one or both of:objective and subjective quantification of how one or more of: facility,machinery and an equipment item operated, which may be depicted in anAVM.

“Design Feature” as used herein, shall mean a value for a variabledescriptive of a specific portion of a Commercial Property. A DesignFeature may include, for example, a size and shape of a structuralelement or other aspect, such as a doorway, window or beam; a materialto be used, an electrical service, a plumbing aspect, a data service,placement of electrical and data outlets; a distance, a length, a numberof steps; an incline; or other discernable value for a variableassociated with a structure or Commercial Property feature.

“Digital Sensor” as used herein includes a Sensor operative to quantifya state in the physical world in a digital representation.

“Experiential Data” as used herein shall mean data captured on orproximate to a subject Processing Facility descriptive of a conditionrealized by the Processing Facility. Experiential data is generated byone or more of: digital and/or analog sensors, transducers, imagecapture devices, microphones, accelerometers, compasses and the like.

“Experiential Sensor Reading” as used herein shall mean a value of asensor output generated within or proximate to a subject ProcessingFacility descriptive of a condition realized by the Processing Facility.An Experiential Sensor Reading may be generated by one or more of:digital and/or analog sensors, transducers, image capture devices,microphones, accelerometers, compasses and the like.

“Image Capture Device” or “Scanner” as used herein refers to apparatusfor capturing digital or analog image data, an Image capture device maybe one or both of: a two dimensional camera (sometimes referred to as“2D”) or a three dimensional camera (sometimes referred to as “3D”). Insome examples an Image Capture Device includes a charged coupled device(“CCD”) camera.

“Lag Benefit” as used herein shall mean a benefit derived from, or inrelation to a Lead Action.

“Lead Actions” as used herein shall mean an action performed on, in, orin relation to a Commercial Property to facilitate attachment of anPerformance Level.

“Performance” as used herein may include a metric of an action orquantity. Examples of Performance may include metrics of: number ofprocesses completed, energy efficiency; length of service; cost ofoperation; quantity of goods processed or manufacture; quality of goodsprocessed or manufacture; yield; and human resources required.

“Performance Level” as used herein shall mean one or both of a quantityof actions executed and a quality of actions.

“Processing Facility” as used herein shall mean a structure “QualityLevel” capable of receiving in a processing material and/or a consumableand outputting a commercial product.

“Sensor” as used herein refers to one or more of a solid state,electro-mechanical, and mechanical device capable of transducing aphysical condition or property into an analogue or digitalrepresentation and/or metric.

“Smart Device” as used herein includes an electronic device including,or in logical communication with, a processor and digital storage andcapable of executing logical commands.

“Total Resources” as used herein shall mean an aggregate of one or moretypes of resources expended over a time period.

“Vantage Point” as used herein refers to a specified location which maybe an actual location within a physical facility or a virtualrepresentation of the actual location within a physical facility.

“Virtual Processing Facility” (“VPS”): as used herein shall mean adigital representation of a physical structure suitable for commercialuse. The Virtual Processing Facility may include Design Features and AsBuilt Features. The Virtual Processing Facility may be included as partof an AVM.

Referring now to FIG. 1A a block diagram illustrates various aspects ofthe present invention and interactions between the respective aspects.The present invention includes an Augmented Virtual Model 111 of aProcessing Facility that includes As Built Features. The generation andinclusion of As Built Features, based upon location and directionspecific data capture, is discussed more fully below. Data may betransmitted and received via one or both of digital and analogcommunications, such as via a wireless communication medium 117.

According to the present invention, one or more Deployment PerformanceMetric 112 are entered into automated apparatus in logical communicationwith the AVM 111. The Deployment Performance Metric 112 may essentiallyinclude a purpose to be achieved during Deployment of a modeledProcessing Facility. By way of non-limiting example, a DeploymentPerformance Level may include one or more of: a production or quantity;quality; yield; scalability; a level of energy efficiency; a level ofwater consumption; mean time between failure for equipment included inthe Processing Facility; mean time between failure for machineryinstalled in the structure; a threshold period of time between repairson the Processing Facility; a threshold period of time between upgradesof the Processing Facility; a target market value for a CommercialProperty; a target lease or rental value for a Commercial Property; acost of financing for a Commercial Property; Total Cost of ownership ofa Commercial Property; Total Cost of Deployment of a Commercial Propertyor other quantifiable aspect.

In some embodiments, Deployment Performance Metrics may be related to afungible item, such as a measurement of energy (KWH of electricity,gallon of fuel oil, cubic foot of gas, etc.); man hours of work; trademedium (i.e. currency, bitcoin, stock, security, option etc.); parts ofmanufactures volume of material processed or other quantity. Relatingmultiple disparate Deployment Performance Metrics to a fungible itemallows disparate Performance Metrics to be compared for relative value.

Modeled Performance Levels 113 may also be entered into the automatedapparatus in logical communication with the AVM 111. The ModeledPerformance Levels 113 may include an appropriate level of Performanceof an aspect of the structure in the AVM affected by the DeploymentPerformance Metric 112. For example, a Performance Level 113 for energyefficiency for a structure modeled may include a threshold of KW hoursof electricity consumed by the structure on a monthly basis. Similarly,a target market value or lease value may be a threshold pecuniaryamount. In some embodiments, a pecuniary amount may be according to aperiod of time, such as monthly, or a term of years.

Empirical Metrics Data 114 may be generated and entered into theautomated apparatus on an ongoing basis. The Empirical Metrics Data 114will relate to one or more of the Deployment Performance Metrics and maybe used to determine compliance with a Deployment Performance Leveland/or a Performance Levels. Empirical Metrics Data 114 may include, byway of non-limiting example, one or more of: a unit of energy; an unitof water; a number of service calls; a cost of maintenance; a cost ofupgrades; equipment details, design details, machinery details,identification of human resources deployed; identification oforganizations deployed; number of human resources; demographics of humanresources (i.e. age, gender, occupations, employment status, economicstatus, requiring assistance with basic living necessities; and thelike); percentage of time structure is occupied; purpose of occupancy(i.e. primary residence, secondary residence, short term rental, longterm lease, etc.); Sensor readings (as discussed more fully below); manhours required for structure repair/maintenance/upgrades; total currency(or other fungible pecuniary amount) expended on behalf of a structureor property.

In addition to Empirical Metrics Data 114, Lead Actions and expected LagBenefits 115 that may cause an effect on one or both of a DeploymentPerformance Level 112 and a Performance Level 113, may be entered intothe automated apparatus. A Lead Action may include an action expected toraise, maintain or lower an Empirical Metrics Data 114. For example, anaction to install water efficient plumbing fixtures may be scheduled inorder to improve water consumption metrics. Similar actions may relateto electrically efficient devices, or automatic electric switches beinginstalled; preventive maintenance being performed; structure automationdevices being installed and the like. Other Lead Actions may includelimiting a demographic of occupants of a structure to a certaindemographic, such as senior citizens. An expected benefit may bemeasured in Lag Benefit measurements, such as those described asEmpirical Metrics Data 114, or less tangible benefits, such as occupantsatisfaction.

The automated apparatus may also be operative to calculate FuturePerformance 116 based upon one or more of: AVM Model with As Built Data111; Deployment Performance Metrics 112; Modeled Performance Levels 113and Empirical Metrics Data 114. Future Performance may be calculated interms of an appropriate unit of measure for the aspect for whichPerformance is calculated, such as, for example: an energy unit; manhours; mean time between failures and dollar or other currency amount.

Calculation of Future Performance 116 may be particularly useful tocalculate Total Resources calculated to be required to support aparticular structure, group of structures, properties and/or group ofproperties over a term of years (“Total Resources Calculated”). TotalResources Calculated may therefore be related to calculations of FuturePerformance 116 and include, for example, one or more of: energy units;water units; man hours; equipment; machinery and dollars (or othercurrency or fungible item). In some embodiments, calculations of FuturePerformance may include a Total Cost of Ownership for a term of years.For example, a Total Cost of Ownership for a Commercial Property mayinclude a purchase amount and amounts required for maintenance, repairand upgrades from day one of Deployment through twenty years ofDeployment (a shorter or longer term of years may also be calculated).

Accordingly, some embodiments may include a calculation of TotalResources required that includes a purchase price of a property with aProcessing Facility, that incorporates a total cost associated with theproperty over a specified term of years. The total cost will be basedupon the AVM with As Built Data 111; Deployment Performance Metrics 112;Modeled Performance Levels 113 and Empirical Metrics Data 114.

Moreover, Total Resources required may be aggregated across multipleproperties and. Structures. Aggregation of properties may be organizedinto property pools to mitigate risk of anomalies in the Calculation ofFuture Performance. Of course the benefits of property ownership and/ormanagement may also be pooled and compared to the Total Resourcesrequired. In various embodiments, different aspects of calculated FuturePerformance 116 may be aggregated and allocated to disparate parties.For example, first aggregation may relate to man hours of techniciantime for structure repair and maintenance and the fulfillment ofobligations related to the aggregation may be allocated to a firstparty. A second aggregation may relate to machinery Performance andobligations allocated to a second party. A third aggregation may relateto equipment Performance and obligations allocated to a third party.Other aggregations may similarly be allocated to various parties. Insome embodiments, financial obligations incorporating one or both ofacquisition cost and ongoing Deployment costs may be allocated andfinanced as a single loan. Other embodiments include a calculated FuturePerformance cost being incorporated into a purchase price.

An important aspect of the present invention includes definition andexecution of Lead Actions based upon one or more of: the AVM Model withAs Built Data 111; Deployment Performance Metrics 112; ModeledPerformance Levels 113; Empirical Metrics Data 114 and Calculations ofFuture Performance 116.

Referring now to FIG. 1B, an AVM is generally associated with aCommercial Property that includes a real estate parcel 110-113.According to some embodiments, one or more of an improvement, a repair,maintenance and an upgrade are performed on the Commercial Property. TheCommercial Property is identified according to an automateddetermination of a location and a particular position, elevation anddirection are further determined automatically within the CommercialProperty. Smart Devices may be used to access data records stored in anAVM according to a unique identifier of a physical location of the realestate parcel 110-113.

As illustrated, a map of real estate parcels 110-113 is shown with icons110A-111A indicating parcels 110-111 that have virtual structures110A-111A included in a virtual model associated with the parcels. Otherparcels 113 have an indicator 113A indicating that a virtual model is inprocess of completion.

In some methods utilized by the present invention, data in an AVM may beaccessed via increasingly more accurate determinations. A first level ofgeospatial location determinations may be based upon a real estateparcel 110-113 and a second geospatial determination may be madeaccording to position locators (discussed more fully below) includedwithin the boundaries of the real estate parcel 110-113. Still moreaccurate location position may be calculated according to one or both ofa direction determination and an accelerometer. Accordingly, it iswithin the scope of the present invention to access a record of a designmodel for a specific wall portion within a structure based uponidentification of a real estate parcel 110-113 and a location within astructure situated within the real estate parcel 110-113 and height anddirection. Likewise the present invention provides for accessing AsBuilt data and the ability to submit As Built data for a specificportion of a structure based upon an accurate position and directiondetermination.

In some implementations of the present invention, a Commercial Propertyunique identifier may be assigned by the AVM and adhere to a standardfor universally unique identifiers (UUID), other unique identifiers maybe adopted from, or be based upon, an acknowledged standard or value.For example, in some embodiments, a unique identifier may be based uponCartesian Coordinates, such as global positioning system (GPS)coordinates. Other embodiments may identify a Commercial Propertyaccording to one or both of: a street address and a tax map numberassigned by a county government of other authority.

In some embodiments, an AVM may also be associated with a larger groupof properties, such as a manufacturing plant, research and development,assembly, a complex, or other defined arrangement.

As illustrated, in some preferred embodiments, an electronic recordcorrelating with a specific Commercial Property may be identified andthen accessed based upon coordinates generated by a GPS device, or otherelectronic location device. The GPS device may determine a location andcorrelate the determined location with an AVM record listing model data,As Built data, improvement data, Performance data, maintenance data,cost of operation data, return on investment data and the like.

Referring now to FIG. 1C, a relational view of an Augmented VirtualModel 100 with a Virtual Processing Facility 102B is illustrated. TheAugmented Virtual Model 100 includes a virtual model stored in digitalform with a design aspect that allows for a physical structure 102Asuitable for commercial use to be designed and modelled in a virtualenvironment. The design aspect may reference Performance data offeatures to be included in a Virtual Processing Facility 102B and alsoreference variables quantifying an intended use of the VirtualProcessing Facility 102B. The Virtual Processing Facility 102B and theAugmented Virtual Model 100 may reside in a virtual setting viaappropriate automated apparatus 108. The automated apparatus 108 willtypically include one or more computer servers and automated processorsas described more fully below and may be accessible via known networkingprotocols.

In correlation with the design aspect, the present invention includes anAs Built Model 101 that generates a Virtual Processing Facility 102A inthe context of the Augmented Virtual Model 100. The As Built Model 101includes virtual details based upon As Built data captured on orproximate to a physical status of a related commercial physicalstructure 102A. The As Built data may be captured, for example, duringconstruction or modification of a physical structure 102A.

The As Built Model 101 may include detailed data including imagecaptures via one or more image capture devices 107 and physicalmeasurements of features included in the physical structure 102A. Thephysical measurements may be during a build phase of the physicalstructure; or subsequent to the build phase of the physical structure.In some embodiments, original As Built measurements may be supplementedwith additional data structure data associated with repairs orimprovements are made to the physical structure. Details of recordablebuild aspects are placed as digital data on a recordable medium 104included in the automated apparatus 108.

The digital data included on a recordable medium 104 may thereforeinclude, for example, one or more of: physical measurements capturingExperiential Data; image data (i.e. digital photos captured with a CCDdevice); laser scans; infra-red scans and other measurement mediums. Oneor more records on the recordable medium 104 of an As Built structuremay be incorporated into the Augmented Virtual Model 100 therebymaintaining the parallel nature of the Augmented Virtual Model 100 withthe physical structure 102A.

In some embodiments, As Built data on a recordable medium 104 may begenerated and/or captured via an image capture device 117.

As the physical structure is deployed for use, subsequent measurementsthat generate and/or capture Experiential Data may be made andincorporated into the Augmented Virtual Model 100. In addition, a usermay access and update 103 the Augmented Virtual Model 100 to ascertainfeatures of the physical structure 102A that have been virtuallyincorporated into the Augmented Virtual Model 100. In some examples, atablet, handheld network access device (such as, for example a mobilephone) or other device with automated location service may be used todetermine a general location of a physical structure 102A. For example,a smart phone with global positioning system (GPS) capabilities may beused to determine a physical address of a physical structure, such as123 Main Street. Stored records containing data relating to 123 MainStreet may be accessed via the Internet or other distributed network.

In addition to the use of GPS to determine a location of a User Device,the present invention provides for a real estate parcel with a physicalstructure 102A that includes more radio frequency (or other mechanism)location identifiers 109. Location identifiers 109 may include, forexample, radio transmitters at a defined location that may be used toaccurately identify via triangulation, a position of a user device 106,such as a: tablet, smart phone or virtual reality device. The positionmay be determined via triangulation, single strength, time delaydetermination or other process. In some embodiments, triangulation maydetermine a location of a user device within millimeters of accuracy.

Other location identifiers may include, by way of non-limiting example,RFID chips, a visual markings (i.e. a hash tags or barcode), pins orother accurately placed indicators. Placement of the locationidentifiers may be included in the AVM and referenced as the location ofthe physical user device is determined. As described above, specificlocation identifiers may be referenced in the context of GPS coordinatesor other more general location identifiers.

Based upon the calculated location of the user device 106, details ofthe physical structure 102A may be incorporated into the VirtualProcessing Facility 102B and presented to a user via a graphical userinterface (GUI) on the user device 106.

For example, a user may approach a physical structure and activate anapp on a mobile user device 106. The app may cause the user device 106to activate a GPS circuit included in the user device and determine ageneral location of the user device 106, such as a street addressdesignation. The general location will allow a correct AVM 104B to beaccessed via a distributed network, such as the Internet. Once accessed,the app may additionally search for one or more location identifiers 109of a type and in a location recorded in the AVM. An AVM may indicatethat one or more RFID chips are accessible in a kitchen, a living roomand each bedroom of a structure. The user may activate appropriateSensors to read the RFID chips and determine their location. In anotheraspect, an Augmented Virtual Model 100 may indicate that locationidentifiers 109 are placed at two or more corners (or other placement)of a physical structure 102A and each of the location identifiers 109may include a transmitter with a defined location and at a definedheight. The user device 106, or other type of controller, may thentriangulate with the location identifiers 109 to calculate a preciselocation and height within the physical structure.

Similarly, a direction may be calculated via a prescribed movement ofthe user device 106 during execution of code that will record a changein position relative to the location identifiers 109. For example, auser smart device, such as a smart phone or user device 106 may bedirected towards a wall or other structure portion and upon execution ofexecutable code, the smart device may be moved in a generally tangentialdirection towards the wall. The change in direction of the user device106 relative to the location identifiers 109 may be used to calculate adirection. Based upon a recorded position within the structure 102A andthe calculated direction, a data record may be accessed in the AugmentedVirtual Model 100 and a specific portion of the Augmented Virtual Model100 and/or the Virtual Processing Facility 102B may be presented on theuser device 106. In other embodiments, a direction may be made, orverified via a mechanism internal to the smart device, such as a compassor accelerometer.

In still another aspect of the present invention, in some embodiments,transmissions from one or more location identifiers 109 may becontrolled via one or more of: encryption; encoding; passwordprotection; private/public key synchronization or other signal accessrestriction. Control of access to location identifiers 109 may be usefulin multiple respects, for example, a location identifier mayadditionally function to provide access to data, a distributed networkand/or the Internet.

The Virtual Processing Facility 102B may include one or both of:historical data and most current data relating to aspects viewable orproximate to the user device 106 while the user device is at thecalculated location in the physical structure 102A. In this way, theparallel virtual world of the Augmented Virtual Model 100 and theVirtual Processing Facility 102B may present data from the virtual worldthat emulates aspects in the physical world, and may be useful to theuser accessing the user device 106, while the user device is at aparticular physical location. As discussed within this document, datapresented via the Augmented Virtual Model 100 may include one or moreof: design data, As Built data, Experiential Data, Performance datarelating to machinery and/or features of the Augmented Virtual Model 100or physical structure; maintenance data, and annotations.

Annotations may include, for example, a user's or designer's noterecorded at a previous time, a service bulletin, maintenance log,operation instructions or a personal note to a subsequent user, such asa virtual “John Smith was here” such guest log indicating who hadfrequented the location. Annotations may include one or both of text andimage data. For example, an annotation may include an image of thelocation captured at a given time and date. The image may be of apersonal nature, i.e. the living room while the Smith's owned thestructure, or a professional nature, i.e. the living room after beingpainted by XYZ Contractor on a recorded date. In some embodiments,annotations may be used to indicate completion of a work order.Recordation of completion of a work order may in turn trigger a paymentmechanism for paying an entity contracted to complete the work order. Inanother aspect, annotations may relate to an AVM or a Virtual ProcessingFacility as a whole, or to a particular aspect that is proximate to alocation of the user device within the Virtual Processing Facility.

In some embodiments, details of a proposed use of a structure and parcelmay be input into a design module and used to specify or recommendfeatures to be included in an Augmented Virtual Model 100.

According to the present invention, features of a Processing Facilityand parcel are generated within a digital design model and then trackedas the features are implemented in a build process and further trackedin Performance of the structure as it is placed into use. To the extentavailable, Performance is tracked in the context of variables relatingto use. Variables may include, for example: a use of the structure, suchas manufacturing and/or processing; a number of resources accessing in astructure; demographics of the human resources; number of months peryear the structure is deployed for use; which months of the year astructure is deployed for use; which hours of the day the structure isoccupied and other relevant information.

As Experiential Sensor Readings are generated they may be memorializedto generate Experiential Data associated with a physical structure 102A.The Experiential Data is collected and analyzed via structured queriesand may also be analyzed with Artificial Intelligence processes such asunstructured queries to derive value. In some embodiments, ExperientialData may also be associated with a human and/or an animal interactingwith the structure 102A. Whereas former process plants were generallydesigned and built to mitigate against variability in a human 118 andbetween disparate humans 118. The present invention allows for humanvariability to be monitored via sensors 119 and the structure to bemodified to optimally inter-relate with the values for variablesattributable to a human 118 that will inhabit or otherwise interact withthe structure 102A. Human (and/or animal) maybe quantified with sensors119 installed on or proximate to the Human 118. Alternatively, sensors117 located in, or proximate to, a structure 102A may be used to monitorhuman variability. Biosensors may be used to provide empirical data ofhumans 118 interacting with a structure may be analyzed using structuredor unstructured queries to device relationships between structureperformance and human biometrics. Accordingly, sensors may be used toquantify interaction between a human 118 and an As Built structure 102Aaccording to physiological and behavioral data, social interactions, andenvironmental factors within the structure, actions undertaken,movements, and almost any quantifiable aspect.

As Built Features and biometrics may be further utilized to controlvarious structure automation devices. Structure automation devices mayinclude, by way of non-limiting example one or more of: automated locksor other security devices; thermostats, lighting, heating, chemicalprocessing, cutting, molding, laser shaping, 3D printing, assembly,cleaning, packaging and the like. Accordingly, a structure with recordedAs Built design features and vibration sensors may track activities in astructure and determine that a first occupant associated with a firstvibration pattern of walking is in the structure. Recorded vibrationpatterns may indicate that person one is walking down a hallway andautomatically turn on appropriated lighting and adjust one or more of:temperature, sound and security. Security may include locking doors forwhich person one is not programmed to access. For example, a firstpattern of vibration may be used to automatically ascertain that aperson is traversing an area of a structure for which a high level ofsecurity is required or an area that is designated for limited accessdue to safety concerns. As Built data has been collected. Otherstructure automation may be similarly deployed according to As Builtdata, occupant profiles, biometric data, time of day, or othercombination of available sensor readings.

Referring now to FIG. 1D, according to the present invention a virtualmodel 120 is generated that correlates with a physical facility 120 andincludes virtual representations of As Built features and ExperientialData. As discussed more fully herein, the virtual model may include anAVM 111 with As Built data, such as image data and measurements,included within the model. In addition, sensor data may be collectedover time and incorporated into the AVM 111. The AVM 111 may includevirtual representations of one or more of: sensors 125; equipment126-128; controls 121; infrastructure 129, such as HVAC, utilities, suchas electric and water, gas lines, data lines, etc. and vantage points121.

In some implementations, a virtual reality headset may be worn by a userto provide an immersive experience from a vantage point 121 such thatthe user will experience a virtual representation of what it would belike to be located at the vantage point 121 within the facility 122 at aspecified point in time. The virtual representation may include acombination of Design Features, As Built Data and Experiential Data. Avirtual representation may therefore include a virtual representation ofimage data via the visual light spectrum, image data via infrared lightspectrum, noise and vibration reenactment. Although some specific typesof exemplary sensor data have been described, the descriptions are notmeant to be limiting unless specifically claimed as a limitation and itis within the scope of this invention to include a virtualrepresentation based upon other types of captured sensor data may alsobe included in the AVM 111 virtual reality representation.

Referring now to FIG. 1E, a user 131 is illustrated situated within anAVM 111. The user 131 will be virtually located at a Vantage Point 137and may receive data 136, including, but not limited to one or more of:image data 134, audio data 135 and Ambient Data 136. The user 131 mayalso be provided with controls 133. Controls 133 may include, forexample, zoom, volume, scroll of data fields and selection of datafields. Controls may be operated based upon an item of Equipment 132within a Field of View 138 of the User 131 located at a vantage point137 and viewing a selected direction (Z axis). The user is presentedwith Image Data from within the AVM 111 that includes As Built data andvirtual design data.

Additional examples may include sensor arrays, audio capture arrays andcamera arrays with multiple data collection angles that may be complete360 degree camera arrays or directional arrays, for example, in someexamples, a sensor array (including image capture sensors) may includeat least 120 degrees of data capture, additional examples include asensor array with at least 180 degrees of image capture; and still otherexamples include a sensor array with at least 270 degrees of imagecapture. In various examples, data capture may include sensors arrangedto capture image data in directions that are planar or oblique inrelation to one another.

Referring now to FIG. 2, a functional block illustrates variouscomponents of some implementations of the present invention. Accordingto the present invention automated apparatus included in the AVM 201 areused to generate a model of a Virtual Processing Facility (“VPS”) andmay also incorporate a model and associated real estate parcel (“VPS”).One or more pieces of equipment that will be deployed in the CommercialProperty may be included into the augmented virtual model 201, equipmentmay include, for example: machinery 222; building support items 212, andutilities support 213. The AVM 201 may model operational expectations204 during deployment of a facility and associated machinery andequipment included in the AVM 201. Machinery 211 may include, forexample, manufacturing tools, robots or other automation, transporttools, chemical processing machine, physical processing machine,assembly machine, heat processing machine, cooling machine, depositiondevice, etching device, welding apparatus, cutting apparatus, formingtool, drilling tool, shaping tool, transport machine, structureautomation, air purification or filter systems, noise containment deviceand the like. Utility support equipment may include cabling, dishantennas, Wi-Fi, water softener, water filter, power, chemical supply,gas supply, compressed air supply and the like, as well as uptime anddowntime associated with a facility utility 243.

The AVM 201 calculates a predicted Performance of the AVM and generatesOperational Levels 204 based upon the Performance 222, wherein“Performance” may include one or more of: total cost of deployment 214;operational experience 203 which may include one or both of: objectiveempirical measurements and satisfaction of operator's use an As Builtphysical model based upon the AVM; operational expectations 204, totalmaintenance cost 206, and residual value of an As Built following a termof years of occupation and use of an As Built Facility based upon theAVM. Performance 221 may also be associated with a specific item ofmachinery 211.

In another aspect, actual Operational Experience 203 may be monitored,quantified and recorded by the AVM 201. Data quantifying the OperationalExperience 203 may be collected, by way of non-limiting example, fromone or more of: Sensors incorporated into an As Built structure;maintenance records; utility records indicating an amount of energy 202(electricity, gas, heating oil) consumed; water usage; periodicmeasurements of an As Built structure, such as an infra-red scan ofclimate containment, air flow through air handlers, water flow, waterquality and the like; user surveys and maintenance and replacementrecords.

In still another aspect, a warranty 205 covering one or both of partsand labor associated with an As Built structure may be tracked,including replacement materials 207. The warranty 205 may apply to anactual structure, or one or more of machinery 211; building support item212; and utility support item 213.

The AVM 201 may take into account a proposed usage of a Deployment of aProcessing Facility based upon values for Deployment variables, andspecify aspects of one or more of: Machine s 211; building support 212;and utility support 213 based upon one or both of a proposed usage andvalues for Deployment variables. Proposed usage may include, forexample, how many human resources will occupy a Processing Facility,demographics of the resources that will occupy the Processing Facility;percentage of time that the Processing Facility will be occupied,whether the Processing Facility is a primary residence, whether theProcessing Facility is a leased property and typical duration of leasesentered into, environmental conditions experienced by the ProcessingFacility, such as exposure to ocean salt, Winter conditions, desertconditions, high winds, heavy rain, high humidity, or other weatherconditions.

In another aspect, Deployment may relate to biometrics or other dataassociated with specific occupants of a structure. Accordingly, in someembodiments, sensors may monitor biologically related variables ofoccupants and/or proposed occupants. The biometric measurements may beused to determine one or both of Lead Actions and Lag Metrics. Leadactions may include one or more of: use of specific building materials,selection of design aspects; Deployment of structure equipment;Deployment of machinery; terms of a lease; length of a lease: terms of amaintenance contract; and structure automation controls.

According to the present invention, design aspects and structurematerials 210 may also be based upon the proposed usage and values forDeployment variables. For example, a thicker exterior wall with higherinsulation value may be based upon a structures location in an adverseenvironment. Accordingly, various demographic considerations andproposed usage of a structure may be used as input in specifying almostany aspect of a Processing Facility.

Total Cost of Deployment (TCD)

In still another consideration, a monetary value for one or more of: aTotal Cost of Deployment (“TCD”). Total maintenance cost (“TMC”) and adesired return on investment (“ROI”) for a Commercial Property may beused as input for one or more design aspects included in an AugmentedVirtual Model System 200. Total Cost of Ownership, Total MaintenanceCost and ROI may be used to determine optimal values of variables202-205, 210-213 specified in an Augmented Virtual Model System 200 andincorporated into an As Built structure, and other improvements to areal estate parcel.

A Total Cost of Deployment 214 may change based upon a time period 215used to assess the Total Cost of Deployment 214. A ROI may include oneor more of: a rental value that may produce a revenue stream, a resalevalue, a cost of operation, real estate taxes based upon structurespecifications and almost any other factor that relates to one or bothof a cost and value.

Desirable efficiency and Performance may be calculated according to oneor more of: established metrics, measurement protocols and pastexperience. The AVM 201 and associated technology and software may beused to support a determination of a TCD. In another aspect, a TCD maybe based upon an assembly of multiple individual metrics, procedures toassess metrics, procedures to adjust and optimize metrics and proceduresto apply best results from benchmark operations. In the course ofmanaging Total Cost of Ownership, in some examples, initial steps mayinclude design aspects that model an optimal design based upon TotalCost of Ownership metrics and also model designed algorithms used toassess Total Cost of Ownership metrics.

In the following examples, various aspects of Total Cost of Deployment214, Total Maintenance Costs, and associated metrics, are considered inthe context of calculating a target Total Cost of Deployment 214.Accordingly, the AVM may be used to TCD optimization.

A designed Processing Facility is ultimately built at a site on a realestate parcel. A build process may be specified and provide metrics thatmay be used in a process designed by a AVM 201 and also used as aphysical build proceeds. In some examples, time factors associated witha physical build may be important, and in some examples time factorsassociated with a physical build may be estimated, measured and actedupon as they are generated in a physical build process. Examples of timefactors may include, one or more of: a time to develop and approve siteplans; a time to prepare the site and locate community providedutilities or site provided utilities; a time to lay foundations; a timeto build structure; a time to finish structure; a time to installinternal utilities and facilities related aspects; a time to install,debug, qualify and release equipment; times to start production runs andto certify compliance of production are all examples of times that canbe measured by various techniques and sensing equipment on a ProcessingFacility's site. Various time factors for a build are valuable and maybecome increasingly valuable as a physical build proceeds since themonetary investment in the project builds before revenue flows andmonetary investments have clearly defined cost of capital aspects thatscale with the time value of money.

Various build steps may include material flows of various types.Material flow aspects may be tracked and controlled for cost andefficiency. Various materials may lower a build materials cost, butraise time factors to complete the build. Logical variations may becalculated and assessed in an AVM 201 and optimal build steps may begenerated and/or selected based upon a significance placed upon variousbenefits and consequences of a given variable value. Physical buildmeasurements and/or sensing on physical build projects may also be usedas input in an assessment of economic trade-offs.

The equipment deployed may incur a majority of a build cost dependingupon user defined target values. The AVM may model and presentalternatives including one or more of: cost versus efficiency, quality240, time to build, life expectancy, market valuation over time. A costto build may be correlated with cost to deploy and eventual resale. Anoverall model of a Total Cost of Deployment 214 may include any or allsuch aspects and may also include external. In some examples, the natureof equipment trade-offs may be static and estimations may be made fromprevious results. In some other examples, changes in technology,strategic changes in sourcing, times of acquisition and the like mayplay into models of Total Cost of Deployment 214.

In some examples, an initial efficiency of design which incurs largecosts at early stages of a project may have a dominant impact on TotalCost of Deployment 214 when time factors are weighted to real costs. Inother examples, the ability of a Processing Facility to be flexible overtime and to be changed in such flexible manners, where such changes areefficiently designed may dominate even if the initial cost aspects maybe less efficient due to the need to design in flexibility. As aProcessing Facility is built, and as it is operated the nature ofchanging customer needs may create dynamic aspects to estimations ofTotal Cost of Deployment 214. Therefore, in some examples, estimates onthe expected dynamic nature of demands on a Processing Facility may bemodeled against the cost aspects of flexibility to model expectations ofTotal Cost of Deployment 214 given a level of change.

In some examples, factors that may be less dependent on extrinsicfactors, such as product demand and the like may still be importantmetrics in Total Cost of Deployment 214. Included in the As Builtfactors may be calculations such as HVAC temperature load, in whichpersonnel and seasonal weather implications may be important. AVM modelsmay include a user interface to receive value useful in the AVM models.In addition, electronic monitoring, via Sensors that may determineenergy consumption, includes for example: electricity, fuel oil, naturalgas, propane and the like may be useful for estimation and measurement.

Temperatures may be monitored by thermocouples, semiconductor junctionbased devices or other such direct measurement techniques. In otherexamples, temperature and heat flows may be estimated based on photonbased measurement, such as surveying the Processing Facility withinfra-red imaging or the like.

Utility load may be monitored on a Processing Facility wide basis and/orat point of use monitoring equipment located at hubs or individualpieces of equipment itself. Flow meters may be inline, or external topipes wires or conduits. Gases and liquid flows may be measured withphysical flow measurements or sound based measurement. In otherexamples, electricity may be monitored as direct current measurements orinferred inductive current measurement.

In some examples, the nature and design of standard usage patterns of aProcessing Facility and an associated environment may have relevance toTotal Cost of Ownership. For example, usage that includes a largernumber of ingress and egress will expose an HVAC system to increasedload and usage that includes a significant number of waking hours withinhabitants in the commercial building may incur increased usage of oneor more of: machinery 211; building support devices 212; and utilities234.

The nature and measurement aspects of vibration in the ProcessingFacility may also be modelled and designed as the Processing Facility isbuilt. There may be numerous means to measure vibrations from capacitiveand resistive based measurements to optical based measurements thatmeasure a subtle change in distance scale as a means of detectingvibration. Vibration may result from a Processing Facility being locatedproximate to a roadway, train, subway, airport, tidal flow or othersignificant source of relatively consistent vibration. Vibration mayalso be more periodic, such as earthquake activity. In still anotheraspect, vibration may result from human traffic within the CommercialProperty. The use of vibration monitoring Sensors may indicate variousactivities that take place within the structure and facilitate moreaccurate modeling of a life expectancy of various aspects of thestructure as well as machines located within the structure.

Noise levels are another type of vibrational measurement which isfocused on transmission through the atmosphere of the ProcessingFacility. In some cases, noise may emanate from one location aftermoving through solid structure from its true source at another location.Thus, measurement of ambient sound with directional microphones or othermicrophonic sensing types may be used to elucidate the nature andlocation of noise emanations. In some cases, other study of the noiseemanations may lead to establishment of vibrational measurement ofdifferent sources of noise. Floors, ceilings, doorways, countertops,windows and other aspects of a Processing Facility may be monitored inorder to quantify and extrapolate noise levels. Noise and vibrationalmeasurement devices may be global and monitor a region of a ProcessingFacility, or they may be inherently incorporated into or upon individualequipment of the Processing Facility.

In some examples, models of a Processing Facility (including originalmodels and As Built models) may include routings of pipes, wires,conduits and other features of a Processing Facility and the installedequipment that have structure. Together with models of the buildingstructure and the equipment placed in the building the various routedstructures may be married in a detailed AVM 201.

In another aspect, an AVM 201 may include conflicts between the physicalstructures may be detected and avoided in the design stage at farimproved cost aspects. In some examples, a designer may virtuallyascertain a nature of the conflict and alter a design in virtual spaceto optimize operational aspects. Additionally, in some embodiments, anAs Built model may be generated during and after a Processing Facilityis built for various purposes. In some examples, a technician mayinspect a Processing Facility for conformance of the build to thedesigned model. In other examples, as an As Built Processing Facility isaltered to deal with needed changes, changes will be captured andincluded in the As Built AVM 201.

In another aspect of the present invention, the AVM 201 may be used togenerate a virtual reality model of a Commercial Property, including oneor more structures that may be displayed via user interface thatincludes an immersion of the user into a virtual setting. Immersion maybe accomplished, for example, via use of a virtual reality headset withvisual input other than a display screen is limited. In someembodiments, a virtual setting may be generated based upon a location ofthe user. For example, GPS coordinates may indicate a CommercialProperty and a user may wear a headset that immerses the user in avirtual reality setting. The virtual reality setting may display one ormore virtual models of structures that may be potentially constructed onthe Commercial Property.

Embodiments may include models generated, standard modelling softwaresuch as BIM 360™ field which may support the display of a ProcessingFacility design in a very complete level of detail. Modelling of aProcessing Facility in its location or proposed location, or in multipleproposed locations, may be useful from a Total Cost of Ownershipperspective, especially from an evaluation of the nature of a sitelayout including real estate property parcel options and the like.

In some examples, a virtual display observed in the field at the site ofan As Built or proposed build may allow for design changes and designevaluations to be viewed in a space before build is completed. Forexample, a structure may be completed to the extent that walls, floorsand ceilings are in place. A user may utilize a virtual display tounderstand the layout difference for different designs and the designsmay be iterated from designs with the least flexibility to more flexibleyet more complex designs.

In some examples, the design systems may include various types offeatures such as building structure, walls, ducts, utilities, pipes,lighting, and electrical equipment. The design systems are augmentedwith As Built Data and Experiential Data.

The design and modelling systems may be utilized to simulate and projectcost spending profiles and budgeting aspects. The modelling systems maytherefore be useful during the course of an audit, particularly whencomparing actual versus projected spending profiles. The comparison ofvarious spend sequencing may be used to optimize financing costs,maintenance, refurbishing and sequencing. The AVM 201 may be useful toprovide early estimates, and for cost tracking versus projections whichmay be visualized as displays across a virtual display of the building,facilities and equipment.

Energy/Utilities Cost: There may be numerous examples of tradeoffs insources of electric energy to a Processing Facility. For example, a sitemay be designed with various utility supplies for power, with tailoredpower management systems to balance the capacitance and impedance of theeffective load to minimize electricity cost. In addition, variousalternative forms of electric energy may be assessed and designed.Solar, geothermal and Wind generated electric power may make economicsense under certain conditions and may have time of day and seasonalrelevance. The design of flexible support facilities for theinstallation of initial energy generation capacity with provision forthe addition of additional capacity may be assessed. In some instances,backup power generation may be designed to ensure that a ProcessingFacility may run at some level for a certain period of time. In somecases, this may allow for continued production, in other examples,backup power may give a Processing Facility the time to idle and shutdown capacity in a safer and less damaging manner.

In some examples, an energy source for heating, cooling, humidificationand dehumidification equipment may be modelled and managed. In someexamples, a source of energy used may be one or more of electric,natural gas, propane, fuel oil or natural gas. Emergency backup may alsobe modelled and managed. Various choices between electric sources. Solarand fuel based energy consumption may be modelled and controlled basedon upon market forecasts. Estimates may be periodically adjustedaccording to world and/or market events.

Enhanced inspection, and guidance capabilities enabled via ongoingelectronic Sensor measurements may facilitate one or more of:maintenance, expansion and optimization of Processing Facility features,operation Commercial Property equipment and maintenance models. Ongoingmonitoring via Sensor data collection also increases knowledge ofmachines and operations, or other useful capacities towards knowing thestate of the Processing Facility. Decisions related to maintenance ofequipment and facilities may be important decisions that modelling andoperational management systems support. The various cost elements thatmay go into modelling may include, for example, one or more variablesrelated to consumables, such as: a cost of consumables; frequency ofreplacement 241, quantity of consumables 242, life of replaced parts,nature of failures of different part types; manpower associated withplanned and unplanned maintenance and expected and actual life ofequipment

Inside of a functional Processing Facility, augmented reality functionsviewable in an AVM 201 including an AVM may be used to guide operators,surveyors, repair workers, or other individuals, through the ProcessingFacility. As one non-limiting example, a tablet, mobile device, or othersmall device with a screen, imaging, and other sensing capabilities maybe used in an augmented reality fashion towards this function.

As described above, facing a mobile device towards an area in aProcessing Facility and movement of the mobile device in a particularpattern may be used to ascertain a specific area of the ProcessingFacility for which AVM 201 data should be accessed. A combination of oneor more of: image, location, orientation, and other Sensors may also beused to identify to the mobile device, which wall segment, buildingaspect, machinery or equipment the device is identifying. A location ofmobile device, a height and an angle of view may also be utilized todetermine aspects of the structure for which a virtual model is beingrequested.

In some embodiments, a user may be presented with various layers ofdata, including, for example, one or more of: structural aspects of theProcessing Facility, plumbing, electrical, data runs, materialspecifications or other documentation, including but not limited to:basic identifying information, installation information, servicerecords, safety manuals, process records, expected service schedule,among many other possibilities.

A plurality of information may be thus easily accessible inside theProcessing Facility, and may be used for a variety of functions,including finding a specific machine to then diagnose and service aproblem, regular inspection of equipment, guided tours of the ProcessingFacility, or many other functions. This information may be conveyed tothe individual in a plurality of possible formats, such as lists thatshow up on the screen, clickable icons that show up next to theequipment in a Virtual Reality (“VR”) camera feed, or many otherpossibilities. These functions may also be accessible in a hands-free VRformat with a VR headset, or other such device.

As the user is inside a Processing Facility, the user may receive aplurality of information, instructions, etc. while the user is proximateto the various aspects of the structures. For example, the user machinesthemselves, seeing them work, hearing the sounds they make, etc. tobetter inspect or service, among other possible functions, theProcessing Facility's equipment. With VR systems, similar travel,guidance, or inspection capabilities for a functional ProcessingFacility may be achieved completely remotely from the ProcessingFacility itself. Additionally, with VR systems, these capabilities mayoccur prior, during, or after the construction and deployment of aProcessing Facility.

A VR system may constitute a headset or lens system with stereoscopicviewing capabilities, a sound conveying means, such as headphones, andvarious forms of user input, such as a handheld controller or footpedals as non-limiting examples. Various forms of imaging, surveying, ormodeling technology may be used to generate virtual models of afunctional Processing Facility. As a non-limiting example, exploringsuch a model with a VR system may be used to examine layout,functioning, or other parameters of a Processing Facility before itsconstruction. As an alternative non-limiting example, exploring a modelpossibly generated by sensing technology in real time, or over a periodof time prior to viewing with a VR system, may allow for inspection ordemonstration capabilities in a location entirely remotely from theactual Processing Facility itself. This may include both imagery andsounds captured within the Processing Facility.

Collection of data may additionally include actual service lifeexperienced and performance of equipment used in an AVM which therebyenables enhanced modeling of a life expectancy of equipment included inan Augmented Virtual Model 100 and an As Built structure. VariousSensors may gather relevant data related to one or more of: use ofmachinery and equipment, performance of machinery items of equipment andan ambient environment inside or proximate to machinery and equipment.In addition, an unstructured query relating to the functioning or lifeexpectancy of equipment may be generated by a processor to access andinterpret data, thereby deriving relevant input to a decision makerbased upon analysis of the data.

Various examples of data to be acquired, relating to life expectancy ofequipment, may include, but is not limited to, hours of operation,conditions of operation (whether and how long the equipment may berunning under capacity, at rated capacity, or over capacity), or manyenvironmental conditions for operation; environmental conditions mayinclude the ambient temperature (or the difference in ambienttemperature from an ideal or other measured value), ambient humidity (orthe difference in ambient humidity from an ideal or other measuredvalue), ambient air particulate content (or a comparison of the currentair particulate level to a filter change schedule), presence orconcentration of ambient gasses (if relevant) such as carbon dioxide, orother gas, a number of times of ingress or egress into the ProcessingFacility which may change ambient conditions or other trackable data.

Identification of Equipment

Identification capabilities may be facilitated or improved for one ormore of: structural aspects, machinery, equipment and utility supportwithin the Processing Facility. This identification may take many formsthrough various means of query and communication, and may be facilitatedthrough various hardware and/or software means.

Non-limiting examples may include image based identification; a devicewith some imaging means, including but not limited to a mobile devicecamera, tablet device camera, computer camera, security camera, or ARheadset camera may image the equipment to be identified. Imagerecognition software may be used to identify the visualized equipment byits identifying features. Machine learning may be used to train systemsusing this software to identify specific features of the equipment inquestion. Other types of visual identifiers including but not limited toQR codes, may be used to visually identify equipment.

An additional non-limiting example may include location basedidentification; a device with some location means, including but notlimited to GPS, internal dead-reckoning, or other means, may be used todetermine a location within a Processing Facility. Identifyinginformation for equipment at or near the measured location may beaccessed for assessment, based on its proximity to the location basedsignal.

An additional non-limiting example may also include direction basedidentification; with a fixed location, or in tandem with a locationmeans, a device may have capabilities to deduce orientation basedinformation of the device. This orientation information may be used todeduce a direction that the device is pointing in. This direction basedinformation may be used to indicate that the device is pointing to aspecific piece of equipment that may be identified.

An additional non-limiting example may also include As Built sensor andsensor generated experiential data based identification; identifyinginformation for various equipment may be stored and accessed within adatabase storing this information. This information may be accessed byvarious means by a user with certain qualification to that information.

An additional non-limiting example may include tag based identification;identifying information for various equipment may be accessed throughproximity to many non-limiting examples of tagging capabilities, such asmagnetic tags, bar code tags, or others. These tags may contain theinformation in question, or may reference the location of pertinentinformation to the owner, in order to convey this information to theowner.

An additional non-limiting example, data aggregation may include sensorsgenerating data that is associated with an IoT (Internet of Things)based identification. Various IoT devices (or Sensors) may include adigital storage, processor and transmitter for storing and conveyingidentifying information. Upon request, an IoT device may relayidentifying information of itself to a human with a communicatingdevice, or to its neighbors. It may also possibly convey informationreceived from and/or sent to other internet connected devices as well.

Data aggregated and stored for reference in calculation of Cost ofUpkeep considered in a TOC and may include data related to some or allof:

-   -   Documented items covered;    -   Long term warranty for Processing Facility/building ownership;    -   Items included in purchase price;    -   financed amounts;    -   Tax implications;    -   Capital value;    -   Ability to expand Processing Facility and/or structural features        such as baths or kitchens;    -   Lateral dimensions;    -   Vertical dimensions;    -   Building support systems;    -   Utilities;    -   Electric;    -   Water;    -   Discharge;    -   Aggregate Data;    -   Same Processing Facility;    -   Multiple similar facilities;    -   Disparate Processing Facility types;    -   Same geographic area;    -   Disparate geographic areas;    -   Locating Machine s and Equipment;    -   GPS (may be used in combination with other location        technologies;    -   Near field communication with reference point emitter in        Processing Facility;    -   Wi-Fi;    -   RFID;    -   Reflector tags;    -   “Visual” recognition identifiers, i.e. hash, barcode; and    -   Directional—accelerometers in combination with visual        recognition identifiers.

As per the above listing, functionality may therefore include modeledand tracked Performance of a Processing Facility and equipment containedwithin the Processing Facility, including consumables 233 used andtiming of receipt and processing of consumables; modeled and actualmaintenance 232, including quality of maintenance performed; equipmentPerformance including yields; Consumables 233 tracking may include afrequency of replacement and quantity of replaced consumables; Utilities234 tracking may include projected and actually units of energyconsumed.

3D Scanning & Model Development

In one aspect of the present invention data related to the position andidentity of substantial elements of a Processing Facility are firstdesigned and then recorded in their actual placement and installation.This may include locations of building features, such as beams, walls,electrical junctions, plumbing and etc. as the structure is designed andconstructed. As part of the Processing Facility model, laser scanningmay be performed on site at various disparate times during construction.An initial scan may provide general information relating to the locationof the structure in relationship to elements on the property such asroadways, utilizes such as electricity, water, gas and sewer to identifynon-limiting examples.

Additional events for scanning may occur during the construction processin order to capture accurate, three-dimensional (3D) “as-built” pointcloud information. Point cloud may include an array of points determinedfrom image capture and/or laser scanning or other data collectiontechnique of As Built features. In some examples, captured data may beconverted into a 3D model, and saved within a cloud-based data platform.

In some examples other methods of capturing spatially accurateinformation may include the use of drones and optical scanningtechniques which may include high resolution imagery obtained frommultiple viewpoints. Scanning may be performed with light based methodssuch as a CCD camera. Other methods may include infrared, ultraviolet,acoustic, and magnetic and electric field mapping techniques may beutilized.

Processing Facility related information may include physical featuresgenerally associated with an exterior of a structure such asgeo-location, elevation, surrounding trees and large landscapingfeatures, underground utility locations (such as power, water, sewer,sprinkler system, and many other possible underground utility features),paving, and pool or patio areas. Processing Facility related informationmay also include features generally related to a structure such asunderground plumbing locations, stud locations, electrical conduit andwiring, vertical plumbing piping, and HVAC systems or other duct work.The acquisition of the data may allow the model system to accuratelylocate these interior and exterior features. Acquisition of As Builtdata during different points of the construction completion allowsmeasurements to be taken prior to aspects involved in a measurementprocess being concealed by concrete, sheetrock or other various buildingmaterials.

Data is acquired that is descriptive of actual physical features as thefeatures are built and converted into a 3D model which may be referredto as the “As Built” model. The As Built model will include “keycomponents” of the structure and be provided with a level of artificialintelligence that fully describes the key component. In someembodiments, the As Built model may be compared to a design model. Insome implementations “intelligent parameters” are associated with keycomponents within the 3D model. For example, key components andassociated information may further be associated with intelligentparameters. Intelligent parameters for the key components may includethe manufacturer, model number, features, options, operationalparameters, whether or not an option is installed (and if so, itsfeatures and dimensions), any hardware associated with the key component(and its manufacturer and serial number), an owner's manual and servicecontract information, as non-limiting examples. Intelligent parametersassociated with a functional key component such as, HVAC Equipment, mayinclude the manufacturer, model number, capacity, efficiency rating,serial number, warranty start date, motor size, SEER rating, an owner'smanual associated with the equipment, and service contract information.

Key components of the structure may have an identification device suchas a two or three dimensional graphical code (such as a QR code label) aRadio Frequency Identification Chip (RFID) attached that is accessibleto a user, such as a structure owner, structure builder or servicetechnician. When scanned with an apparatus capable of reading the code,a user interface on a display of various types, such as a tablet, mayuse the associated identification, such as a QR code, to provide directaccess to related information. In some examples, the display may showtextual or tabular representations of related data.

In other examples, graphical data such as images, drawings, and the likemay be displayed. In still further examples, both graphical and textualdisplays may be associated with the code. Although a QR code may providean example, other identification technologies such as radio frequencyID, Internet of things (IoT) communication protocols with associatedstored information, and other devices that can receive a signal andrespond with stored information may be used. As well, numerous othertypes of graphical codes in addition to QR code may be read by a deviceand provide a connection between a key component, machinery, locationand other identified aspect and associated data. In some examples, animage based code may be displayed using paints or pigments which are notvisible to the human eye, such as in a non-limiting example ultravioletpigments. In some other examples, a paint or pigment may not be visibleuntil it is made to emit visible light by irradiating it with aparticular band of electromagnetic radiation, such as, for example,ultraviolet light.

In some examples, key components may include doors, windows, masonry,roofing materials, insulation, HVAC equipment and machinery.

An automated Commercial Design and Monitoring (“RDM”) system may supportdynamic updating of tracked aspects. For example, as a structure owneracquires new or additional key components, such as machinery, HVAC,plumbing additions, key components may be added into the As Built modeland the key components may be tracked as a part of the model. Otheraspects may be dynamically updated such as when additions are made tothe building structure or rebuilding of internal structure is made asnon-limiting examples.

Since the As Built model includes information in a database and dynamicmodel functionality exists that commences as a building structure isbeing constructed, the model may assume new support aspects to theconstruction process itself. For example, a benefit from the definitionand utilization of many components within a Processing Facilityutilizing the system herein includes the ability to pre-cut and/orpre-fabricate studs and framing, roofing cuts, masonry, under-slabplumbing, HVAC ductwork, electrical, and other such components. Thedimensions of these various components may be dynamically updated basedon an original model that may be compared to actual fabricated structureas realized on a building site. In some examples a structure builder mayuse a display interface associated with the system and model to displaya comparison of an original set of building plans to a current structureat a point in time which may allow the builder to authorize anystructural changes or variances to design and thereafter allow thedescription of following components to be dynamically adjusted asappropriate. The system may be of further utility to support variousinspections that may occur during a building project which may associatedetected variances with design expert review and approval. An inspectormay be able to utilize the system as allowed on site or operate a windowinto the system from a remote location such as his office.

As the system is utilized during construction, orders for customizedcomponents may be placed. These customized components may be labeled anddelivered to site, in an appropriate sequence, for assembly bycarpenters. This may contribute to a minimization of waste at theworksite, as well as provide a work product that is entirely consistentwith a pre-determined model which may have approved changes that aretracked. The result may improve the quality of the work product, andmake it easier to generate the measured point-cloud 3D model.

Performance Tracking

In another aspect, the AVM system can autonomously and/or interactivelyobtain, store and process data that is provided to it by components ofthe Processing Facility as the structure is built, installed oradditions are made to the structure. The generation, modeling, capture,use, and retention of data relating to Performances in specificequipment or in some cases aspects relating to the design of a facility,may be monitored by the system.

In some examples, Operational Performance may be assessed by processingsampled data with algorithms of various kinds. Feedback of the status ofoperation and of the structure as a whole or in part, as assessed byalgorithmic analysis may be made to a structure owner or a structurebuilder. In addition, a variety of data points gathered via appropriateSensors, visual and sound data may be recorded and stored and correlatedto 3D models of the facility. Experiential Sensor readings may include,by way of non-limiting example: temperature, power usage, utilitiesused, consumables, product throughput, equipment settings, and equipmentPerformance measurement, visual and audible data. Techniques to recorddata points may involve the use of one or more of: electronic Sensors,electro-mechanical Sensors, CCD capture devices, automated inspectionequipment, video camera arrays and audio microphones and arrays of audiomicrophones for the capture and processing of data that may be used togenerate visualizations of actual conditions, either on site or at aremote location. In addition, data may be collected, retained, analyzed,and referenced to project facility Performance.

In some examples, data may also be combined with manufacturer equipmentspecifications and historical data to model expectations related toactual operation of the structure and property aspects.

Virtual Maintenance Support

A 3D model of structure, such as a commercial structure, which may beintegrated with information related to the key components and laserscanned location information, may be made available to the structureowner/structure builder through a computer, an iPad or tablet, or smartdevice. The resulting system may be useful to support virtualmaintenance support.

The three dimensional model may support enhancement to the twodimensional views that are typical of paper based drawings. Althoughthree dimensional renderings are within the scope of informationdelivered in paper format, a three dimensional electronic model mayrender dynamic views from a three dimensional perspective. In someexamples, the viewing may performed with viewing apparatus that allowsfor a virtual reality viewing.

In some examples, a viewing apparatus, such as a tablet or a virtualreality headset, may include orienting features that allow a user suchas a structure owner, structure builder, inspector, engineer, designeror the like to view aspects of a model based upon a location, adirection, a height and an angle of view. A current view may besupplemented with various other information relating to featurespresented in the view. In some examples, the interface may be accessiblethrough a virtual reality headset, computer, or mobile device (such asan iPad, tablet, or phone), as non-limiting examples. Utilizing a deviceequipped with an accelerometer, such as a virtual reality headset ormobile device, as non-limiting examples, a viewable section of the modelmay be displayed through the viewing medium (whether on a screen, orthrough a viewing lens), where the viewer's perspective changes as theaccelerometer equipped device moves, allowing them to change their viewof the model. The viewer's Vantage Point may also be adjusted, through acertain user input method, or by physical movement of the user, asnon-limiting examples.

The presented view may be supplemented with “hidden information”, whichmay include for example, depictions of features that were scanned beforewalls were installed including pipes, conduits, ductwork and the like.Locations of beams, headers, studs and building structure may bedepicted. In some examples, depiction in a view may include asuperposition of an engineering drawing with a designed location, inother examples images of an actual structure may be superimposed uponthe image based upon As Built scans or other recordations.

In a dynamic sense, display may be used to support viewing ofhypothetical conditions such as rerouted utilities, and rebuild wallsand other such structure. In some examples, graphical or text based datamay be superimposed over an image and be used to indicatespecifications, Performance aspects, or other information not related tolocation, shape and size of features in the image.

As presented above, an image may allow for a user to “see through walls”as the augmented reality viewing device simulates a section of a modelassociated with a space displayed via the virtual reality viewingdevice. The viewer's perspective may change as an accelerometer in thevirtual reality viewing device moves. A user may also change a view ofthe AVM, to include different layers of data available in the AVM. Theviewer's Vantage Point may also be adjusted by moving about a physicalspace that is represented by the model. To achieve this, it may bepossible to incorporate positioning hardware directly into a buildingrepresented by the virtual model. The positioning hardware may interfacewith an augmented reality device for positioning data to accuratelydetermine the viewing device's orientation and location with millimeterprecision. The positioning hardware may include, for example a radiotransmitter associated with a reference position and height. Altitude isdifferentiated from height unless specifically referenced since therelative height is typically more important.

Accordingly, a user may access the AVM on site and hold up a smartdevice, such as an iPad or other tablet, and use the smart device togenerate a view inside a wall in front of which the smart device ispositioned, based upon the AVM and the location, height and direction ofthe smart device position.

In some examples, through the use of an augmented reality device, it mayalso be possible to view data, such as user manuals, etc. of associateddevices in the view of a user, simply by looking at them in the viewinginterface. In other examples, there may be interactive means to selectwhat information is presented on the view.

Various electronic based devices implementing of the present inventionmay also be viewed in a virtual reality environment withoutaccelerometer such as a laptop or personal computer. A viewable sectionof a model may be displayed on a Graphical User Interface (GUI) and theviewer's Vantage Point may be adjusted, through a user input device.

The ability to track machinery and other components of a commercialsystem and store the components associated information, such as, forexample user manuals and product specifications and part numbers, mayallow for much more efficient use and maintenance of the componentsincluded within a structure. As well, the system model may also maintainstructure owner manuals and warranties and eliminate the need forstorage and tracking of hard copy manuals.

In a non-limiting example, if a structure owner/structure builderdesires information related to an machinery, it may be found bypositioning a device with a location determining the device within it inproximity to the machinery and accessing the parallel model in theVirtual Processing Facility such as by clicking on the machinery in theVirtual Processing Facility model or by scanning the Code label attachedto machinery. In some examples, an internet of things equipped machinemay have the ability to pair with a user's viewing screen and allow thesystem model to look up and display various information. Thus, the usermay have access to various intelligent parameters associated with thatmachinery such as service records, a manual, service contractinformation, warranty information, consumables recommended for use suchas detergents, installation related information, power hooked up and thelike.

In some examples, an AVM system may include interfaces of various kindsto components of the commercial system. Sensors and other operationalparameter detection apparatus may provide a routine feedback ofinformation to the model system. Therefore, by processing thedata-stream with various algorithms autonomous characterization ofoperating condition may be made. Therefore, the AVM system may provide auser with alerts when anomalies in system Performance are recognized. Insome examples, standard structure maintenance requirements may be sensedor tracked based on usage and/or time and either notification or in somecases scheduling of a service call may be made. In some examples, thealert may be sent via text, email, or both. The structure user may,accordingly, log back into the Virtual Processing Facility to indicatecompletion of a maintenance task; or as appropriate a vendor of suchservice or maintenance may indicate a nature and completion of workperformed.

By detecting operational status, a Virtual Processing Facility may takeadditional autonomous steps to support optimal operation of a commercialsystem. A Virtual Processing Facility may take steps to order andfacilitate shipping of anticipated parts needed for a scheduledmaintenance ahead of a scheduled date for a maintenance event (forexample, shipping a filter ahead of time so the filter arrives prior tothe date it is scheduled to be changed). In another example, a VirtualProcessing Facility may recall notes from an Original EquipmentManufacturer (OEM) that could be communicated to a user through theVirtual Processing Facility. In still further examples, a VirtualProcessing Facility may support a user involved in a real estatetransaction by quantifying service records and Performance of a realproperty.

In still another aspect the AVM may establish a standard maintenance andwarranty program based on manufacturers published data and the abilityto advise structure owners of upcoming needs and/or requirements. Inother examples, the model system may facilitate allowing for structurebuilders, rental companies, or maintenance companies to consolidateinformation for volume discounts on parts or maintenance items. Themodel system may also facilitate minimizing unnecessary time expenditurefor structure builders hoping to minimize needless service calls forwarranty issues, and allowing structure builders and rental companiesattempting to sell a structure or a rental to demonstrate that care hasbeen taken to maintain a structure.

Benefits derived from monitoring and tracking maintenance with a VirtualProcessing Facility may include positively reassuring and educatinglenders and/or lien holders that their investment is being properlycared for. In addition, insurance companies may use access to a VirtualProcessing Facility to provide factual support that their risk isproperly managed. In some examples, a data record in a VirtualProcessing Facility model system and how an owner has cared for theircommercial facility may be used by insurance companies or lenders toensure that good care is being taken. Maintenance records demonstratingdefined criteria may allow insurance companies to offer a structureowner policy discount, such as, for example, installation of an alarmsystem. Additionally, access to a Virtual Processing Facility may allowmunicipalities and utilities to use the info for accurate metering ofutility usage without having to manually check; and peaks in utilitydemand may be more accurately anticipated.

In some examples, Virtual Processing Facility may also be used to assistwith structure improvement projects of various types. In some examples,the structure improvement projects may include support for buildinglarger additions and modifications, implementing landscaping projects.Smaller projects may also be assisted, including in a non-limitingexample such a project as hanging a picture, which may be made safer andeasier with the 3D “as-built” point cloud information. Hidden waterpiping, electrical conduits, wiring, and the like may be located, orvirtually “uncovered”, based on the model database.

Optimization of Commercial Facilities

During construction of a structure corresponding to a Virtual ProcessingFacility, discrete features of the As Built structure may be identifiedvia an identification device such as an IoT device or a QR code label.The ID device may be integrated to the feature or added during the buildscope. Performance monitors may also be simultaneously installed toallow monitoring of Key Performance Indicators (KPIs) for selectedfeatures. In an example, an HVAC system may be added to a commercialfacility during construction and a simultaneously a Performance monitormay be added to the HVAC system. The Performance monitor may be used tomonitor various KPIs for an HVAC system. These KPIs may include outdoorair temperature, discharge air temperature, discharge air volume,electrical current, and the like. Similar monitoring capabilities may beinstalled to all machinery and utilities systems in a commercialfacility. The combination of these numerous system monitors may allowfor a fuller picture of the efficiency of operations of various systems.

Use of the Virtual Processing Facility, which may include data valuescontributed from communication of data from the various monitoringsystems, may allow owners to receive periodic reports, such as in anon-limiting sense monthly emails which may show their current totalenergy consumption as well as a breakdown of what key components arecontributing to the current total energy consumption.

The systems presented herein may be used by owners and facility managersto make decisions that may improve the cost effectiveness of thecommercial system. An additional service for Owners may allow thestructure owner to tap into energy saving options as their structureages. As an example, if a more efficient HVAC system comes on themarket, which may include perhaps a new technology node, the user mayreceive a “Savings Alert”. Such an alert may provide an estimated energysavings of the recommended modification along with an estimate of thecost of the new system. These estimates may be used to generate a reportto the owner of an estimated associated return-on-investment orestimated payback period should the structure owner elect to replacetheir HVAC system.

In some examples, a AVM of a Virtual Processing Facility may set athreshold value for the required ROI above which they may be interestedin receiving such an alert with that ROI is achieved. This informationwill be based on data derived from actual operating conditions andactual historical usage as well as current industry information.Predictive maintenance and energy savings to key systems via SmartStructure Total Cost of Ownership (“TCO”) branded Sensors.

Aggregating Data from Multiple Residences

With the ability to collect and utilize relevant structure informationwith the model system, the aggregation of data and efficiency experiencefrom numerous commercial systems may allow for analysis of optimizationschemes for various devices, machinery and other structure componentsthat includes real installed location experience. Analysis from theaggregated data may be used to provide feedback to equipmentmanufacturers, building materials fabricators and such suppliers.

In some examples, business models may include providing anonymous andaggregated data to original equipment manufacturers as a service modelto give the OEMS an ability to utilize more data to monitor and improvetheir products. In some examples, OEM advertising may be afforded accessthrough the model system. Manufacturers may have an additional sidebenefit motivating the use of this data related to improving theirequipment cost effectives and reliability in order to minimize warrantycost. Such optimized Performance may also provide benefits to bothstructure owners and builders to support their ability to track actualwarranty information, power cost, and overall Performance of astructure.

Methods and Apparatus

Referring to FIGS. 3A-3F, an illustration of the collection of data byscanning a facility during its construction is provided. In FIG. 3A, adepiction of a site for building a facility structure is illustrated.The depiction may represent an image that may be seen from above thesite. Indications of property boundaries such as corners 301 andproperty boarders 302 are represented and may be determined based onsite scanning with property markings from site surveys or may be enteredbased on global coordinates for the property lines. An excavatedlocation 303 may be marked out. Roadways, parking and/or loading areas304 may be located. Buried utilities such as buried telephone 305,buried electric 306, buried water and sewer 307 are located in the modelas illustrated. In some examples, such other site service as a buriedsprinkler system 308 may also be located.

Referring to FIG. 3B the excavated location 303 may be scanned or imagedto determine the location of foundation elements. In some non-limitingexamples, a foundational footing 321 along with buried utilities 322 isillustrated. The buried utilities may include such utilities as electriclines, water supply whether from a utility or a well on location, seweror septic system lines, telecommunications lines such as telephone,cable and internet. Other footing elements 323 may be located atstructural requiring locations as they are built. In some examples ascanning system may provide the locational orientation relative to siteorientation markings. In other examples, aerial imagery such as may beobtained with a drone may be used to convert features to accuratelocation imagery.

Referring to FIG. 3C a wall 331 of the Processing Facility in theprocess of build is illustrated. The structure may be scanned by ascanning element 330. In some examples, a laser three dimensionalscanner may be used. The wall may have supporting features like topplates 333, headers 336, studs 332, as well as internal items such aspipes 334, electrical conduits and wires 335. There may be numerousother types of features within walls that may be scanned as they occursuch as air ducts, data cables, video cables, telephone cables, and thelike.

Referring to FIG. 3D the wall may be completed with structure componentsbehind wall facing 340 may no longer be visible. Electrical outlets 341and door structures 342 may be scanned by a scanning element 330.

Referring to FIG. 3E internal components such as machinery may beinstalled. As a non-limiting example, a machine 350 may be installed andthe resulting three dimensional profiles may be scanned by a scanningelement 330. In some examples, an operational monitor 351 may beattached to the machinery. In some examples, an operational monitor maybe part of the machinery. The operational monitor may have the abilityto communicate 352 data to various receivers that may be connected tothe model system of the residence. In some examples, key structuralcomponents, such as doors, may have identifying devices such as a QRlabel 353. The label may be visible or painted into the structure withnon-visible paint. The identifying devices may provide informationrelated to the device itself and warrantees of the device asnon-limiting examples.

The model may include the various structure elements hidden and visibleand may be used to create output to a display system of a user.Referring to FIG. 3F an example display is illustrated. The variousnon-visible layers may be shown by rendering the covering layers with atransparency. Thus the display shows the machine profile 350 as well asthe internal features that may be concealed like pipes 334, electricalconduits with wires 335, and headers 336 as examples.

Referring to FIG. 3G, an illustration of feedback of the model system isillustrated. A wall that has been scanned with an HVAC unit 360 mayinclude a Performance Monitor 351 which may communication variousinformation wirelessly 352. The communication may be received at anantenna 370 of a router 371 within the commercial facility. Thecommercial facility may be interconnected through the internet 372 to aweb located server 373 which processes the communication. The weblocated server 373 also can include the various model data about thecommercial facility and it can provide composite displays that cansummarize the structure as well as the operational Performance of theHVAC unit 360. It may aggregate the various data into textual andgraphic reports. In some examples it may communicate these reports backthrough internet connections. In other examples, wireless Smart Devicecommunications may be sent to cellular towers 374 which may transmit 375to a Smart Device 376 of a user associated with the commercial facility.

Referring to FIG. 3H an illustration of a virtual reality display inconcert with the present invention is illustrated. A machinery 350 ofthe commercial facility may communicate information to the model server.A user 380 may receive may an integrated communication from the server.The resulting communication may be provided to a virtual reality headset381. The virtual reality headset may provide a display 382 to the userthat provides a three-dimensional view of the physical data as well assimulated imagery that may allow views through objects to hiddenelements behind the object. As well, a heads up type display ofinformation about an object may be superimposed.

Referring now to FIG. 4A, method steps that may be implemented in someembodiments of the present invention are illustrated. At method step401, Deployment aspects may be specified for a Processing Facility andincorporated into a virtual model, such as an AVM discussed above.Deployment aspects may include for example, a purpose for an As Builtstructure that is built based of the AVM. The purpose may include, byway of non-limiting example, one or more of: manufacturing, processing,data processing, health care, research, assembly, shipping andreceiving, prototyping and the like.

Deployment aspects may also include a level of use, such continual,shift schedule or periodic. A climate in which the structure will beplaced may also be considered in the Deployment aspects. Climate mayinclude one or more of: four seasons; primarily winter; tropical,desert; exposed to salt air; and other environmental factors.

At method step 402, a virtual model, such as an AVM is digitally createdaccording to the Deployment aspects of the model. The AVM may includeimprovements to a real estate parcel and a structure that will be placedon the real estate parcel, as well as where a structure may be locatedupon the parcel.

At method step 403, Performance aspects of machinery that may beincluded in the AVM may be digitally modeled and may include a level ofuse of the machinery and an expected satisfaction of the machinery asdeployed according to the Deployment aspects. Maintenance expectations,including a number of repair calls and a preventive maintenance schedulemay also be modeled and associated costs.

At method step 404, Performance aspects of equipment that may beincluded in the AVM may be digitally modeled and may include a level ofuse of the equipment and an expected satisfaction of the machinery asdeployed according to the Deployment aspects. Maintenance expectations,including a number of repair calls and a preventive maintenance schedulemay also be modeled and associated costs.

At method step 405, As Built aspects of a structure are recorded asdiscussed herein, preferably recordation of As Built aspects begins asconstruction begins and continues throughout the existence of thestructure.

At method step 406, the physical structure may be identified via alocation. A physical location may include, for example, CartesianCoordinates, such as Latitude and Longitude coordinates, GPScoordinates, or other verifiable set of location parameters. Inaddition, more exact location specifications may include surveydesignations.

At method step 407, a position within or proximate to the ProcessingFacility may be determined via positioning identifiers. The positionwithin or proximate to the Processing Facility may be determined.

At method step 408, an AVM may be identified and accessed via thephysical location. Once an appropriate AVM is accessed, a particularportion of the AVM may be presented via a GUI based upon the positionwithin the Processing Facility (or proximate to the Processing Facility)and a direction, height and angle of view. The position may bedetermined relative to location identifiers. Height may be determinedvia electronic devices, such as a smart device, or via triangulationreferencing the location identifiers (locations identifiers arediscussed more fully above and below).

At method step 409 an update may be made to a physical ProcessingFacility and at method step 410, the update to the physical structuremay be recorded and reflected in the AVM.

Referring to FIG. 4B, a method flow diagram for commercial monitoringand maintenance is illustrated. At 411 a user may obtain a scanningdevice or devices that may scan a building site. At 412, the user or aservice of the user may mark property boundaries of the commercial site.At 413, work on the commercial site may continue with the excavation ofa building base and the laying down of utilities and other buriedservices. At 414, the scanning device is used to scan the location ofthe various aspects of the building site. At 415, work may continue withthe laying of footings and foundations and other such foundationalbuilding activities. At 416, scanning of the footings and foundationsmay be accomplished. At 417, a commercial structure may be framed andfeatures such as pipe conduit, electrical wiring communications wiringand the like may be added. At 418, the building site may again bescanned to locate the various elements. The framing of the residence maycommence along with running of pipe, wiring, conduits, ducts and variousother items that are located within wall structures. Before coveringsare placed on walls, the framed structure may be scanned at 418.Thereafter, the framed structure may be enclosed with walls 419.

Referring to FIG. 4C a method flow diagram for commercial structuremonitoring and maintenance is illustrated. In this flow diagram, aProcessing Facility may already be built and may have various datalayers already located in the model system. At 421, machinery may beadded to the Processing Facility. At 422, an ID tag, or a QR tag, or andRFID tag or an internet of things device may be associated with themachinery and may be programmed into the model system. At 423, the modelsystem may be interfaced to the machinery ID and into the ProcessingFacility model. At 424, a scanning step may be used to input threedimensional structure data at the installed location into the modelsystem. At 425, an operational monitor function of the device may beadded or activated. At 426, operational data may be transferred from theoperational monitor to the server with the Processing Facility model.

At 427, algorithms running on a server of the model system may determinean operational improvement opportunity based on calculations performedon the data from the operational monitor. At 428 a user may query theoperational data of the machinery for information on its warranty. At429, the model system may initiate an order for a service part and mayschedule a service visit to make a repair based upon analysis of theoperational data. The various steps outlined in the processing flow maybe performed in different orders. In some examples additional steps maybe performed. In some examples, some steps may not be performed.

In some embodiments, the present invention includes a method of trackingattainment of a stated Performance Level relating to a ProcessingFacility, including: a) determining a geographic position of aProcessing Facility via a global positioning system device in a smartdevice proximate to the Processing Facility; b) identifying a digitalmodel of the Processing Facility based upon the geographic position ofthe Processing Facility, the digital model comprising virtualrepresentation of structural components included in the ProcessingFacility; c) referencing multiple positioning reference devices withinthe Processing Facility; d) measuring a distance to at least three ofthe multiple positioning reference devices from a point of measurement;e) calculating a position within the Processing Facility, thecalculation based upon a relative distance of the at least threepositioning reference devices to the point of measurement and atriangulation calculation; f) calculating an elevation of the point ofmeasurement; g) measuring a first state within the Processing Facilitywith a sensor; h) specifying a location of the first state within theProcessing Facility via reference to the position of the point ofmeasurement and the elevation of the point of measurement; i) recordinga first time designation for the step of measuring a first state withinthe Processing Facility with a sensor; and i) correlating the firststate within the Processing Facility and the first time designationattainment of the stated Performance Level.

The geographic position may be calculated with a GPS reading from withinthe Processing Facility. Measuring a distance to the at least three ofthe positioning reference devices may include, one or more of: relativesignal strength received from wireless transmissions emanating from theat least three positioning reference devices; time of arrival of radiosignals of wireless transmissions emanating from the at least threepositioning reference devices measuring a distance to the at least threepositioning reference devices comprises time difference of arrival ofradio signals of wireless transmissions emanating from the at leastthree reference positioning devices.

The above steps may be repeated for at least a second state and a secondtime designation, and in preferred embodiments multiple more states andtime designations.

A state may include, for example, one or more of: a vibration measuredwith an accelerometer; a temperature of at least a portion of thestructure; an electrical current measurement to equipment installed inthe Processing Facility, a number of cycles of operation of equipmentinstalled in the Processing Facility; a number of cycles of operation ofan machinery installed in the Processing Facility; an electrical currentmeasurement to an machinery installed in the Processing Facility; avibration associated with movement of an occupant of the ProcessingFacility.

A vibration pattern may be associated with a specific occupant andtracking the movement of the specific occupant through the structure maybe based upon measured vibration patterns. Similarly, a vibrationpattern may be associated with a particular activity of a specificoccupant and the activity of the specific occupant may be tracked withinthe structure based upon measured vibration patterns.

A Performance Level may include one or more of: operating the ProcessingFacility for a term of years within a threshold use of energy; operatingthe Processing Facility for a term of years within a threshold number ofrepairs; and operating the Processing Facility for a term of yearswithin a threshold budgetary cost.

FIG. 5 illustrates location and positioning identifiers 501-504 that maybe deployed in a Processing Facility according to some embodiments ofthe present invention to determine a user position 500 within orproximate to the Processing Facility 505. Positioning identifiers mayinclude a device that is fixed in a certain location and may be used todetermine via calculation a position of a user with a tablet, smartphone or other network access device able to recognize the positionidentifiers. The position identifiers 501-504 may include devices, suchas, for example, a radio transmitter, a light beacon, or an imagerecognizable device. A radio transmitter may include a router or otherWiFi device. In some embodiments, a position identifier may include aWiFi router that additionally provides access to a distributed network,such as the Internet. Cartesian Coordinates, such as a GPS position 506,may be utilized to locate and identify the Processing Facility 506.

A precise location may be determined via triangulation based upon ameasured distance from three 501-503 or more position identifiers501-504. For example a radio transmission or light signal may bemeasured and compared from the three reference position identifiers501-503. Other embodiments may include a device recognizable via imageanalysis and a camera or other Image Capture Device, such as a CCDdevice, may capture an image of three or more position identifiers501-504. Image analysis may recognize the identification of each ofthree or more of the position identifiers 501-504 and a size ratio ofthe respective image captured position identifiers 501-504 may beutilized to calculate a precise position. Similarly, a heightdesignation may be made via triangulation using the position identifiersas reference to a known height or a reference height.

Referring now to FIG. 6 an automated controller is illustrated that maybe used to implement various aspects of the present invention, invarious embodiments, and for various aspects of the present invention,controller 600 may be included in one or more of: a wireless tablet orhandheld device, a server, a rack mounted processor unit. The controllermay be included in one or more of the apparatus described above, such asa Server, and a Network Access Device. The controller 600 includes aprocessor unit 620, such as one or more semiconductor based processors,coupled to a communication device 610 configured to communicate via acommunication network (not shown in FIG. 6). The communication device610 may be used to communicate, for example, with one or more onlinedevices, such as a personal computer, laptop, or a handheld device.

The processor 620 is also in communication with a storage device 630.The storage device 630 may comprise any appropriate information storagedevice, including combinations of magnetic storage devices (e.g.,magnetic tape and hard disk drives), optical storage devices, and/orsemiconductor memory devices such as Random Access Memory (RAM) devicesand Read Only Memory (ROM) devices.

The storage device 630 can store a software program 640 with executablelogic for controlling the processor 620. The processor 620 performsinstructions of the software program 640, and thereby operates inaccordance with the present invention. The processor 620 may also causethe communication device 610 to transmit information, including, in someinstances, control commands to operate apparatus to implement theprocesses described above. The storage device 630 can additionally storerelated data in a database 650 and database 660, as needed.

Referring now to FIG. 7, a block diagram of an exemplary mobile device702. The mobile device 702 comprises an optical capture device 708 tocapture an image and convert it to machine-compatible data, and anoptical path 706, typically a lens, an aperture or an image conduit toconvey the image from the rendered document to the optical capturedevice 708. The optical capture device 708 may incorporate aCharge-Coupled Device (CCD), a Complementary Metal Oxide Semiconductor(CMOS) imaging device, or an optical Sensor 724 of another type.

A microphone 710 and associated circuitry may convert the sound of theenvironment, including spoken words, into machine-compatible signals.Input facilities may exist in the form of buttons, scroll wheels, orother tactile Sensors such as touch-pads. In some embodiments, inputfacilities may include a touchscreen display.

Visual feedback to the user is possible through a visual display,touchscreen display, or indicator lights. Audible feedback 734 may comefrom a loudspeaker or other audio transducer. Tactile feedback may comefrom a vibrate module 736.

A motion Sensor 738 and associated circuitry convert the motion of themobile device 702 into machine-compatible signals. The motion Sensor 738may comprise an accelerometer that may be used to sense measurablephysical acceleration, orientation, vibration, and other movements. Insome embodiments, motion Sensor 738 may include a gyroscope or otherdevice to sense different motions.

A location Sensor 740 and associated circuitry may be used to determinethe location of the device. The location Sensor 740 may detect GlobalPosition System (GPS) radio signals from satellites or may also useassisted GPS where the mobile device may use a cellular network todecrease the time necessary to determine location. In some embodiments,the location Sensor 740 may use radio waves to determine the distancefrom known radio sources such as cellular towers to determine thelocation of the mobile device 702. In some embodiments these radiosignals may be used in addition to GPS.

The mobile device 702 comprises logic 726 to interact with the variousother components, possibly processing the received signals intodifferent formats and/or interpretations. Logic 726 may be operable toread and write data and program instructions stored in associatedstorage or memory 730 such as RAM, ROM, flash, or other suitable memory.It may read a time signal from the clock unit 728. In some embodiments,the mobile device 702 may have an on-board power supply 732. In otherembodiments, the mobile device 702 may be powered from a tetheredconnection to another device, such as a Universal Serial Bus (USB)connection.

The mobile device 702 also includes a network interface 716 tocommunicate data to a network and/or an associated computing device.Network interface 716 may provide two-way data communication. Forexample, network interface 716 may operate according to the internetprotocol. As another example, network interface 716 may be a local areanetwork (LAN) card allowing a data communication connection to acompatible LAN. As another example, network interface 716 may be acellular antenna and associated circuitry which may allow the mobiledevice to communicate over standard wireless data communicationnetworks. In some implementations, network interface 716 may include aUniversal Serial Bus (USB) to supply power or transmit data. In someembodiments other wireless links may also be implemented.

As an example of one use of mobile device 702, a reader may scan somecoded information from a location marker in a facility with the mobiledevice 702. The coded information may include for example a hash code,bar code, RFID or other data storage device. In some embodiments, thescan may include a bit-mapped image via the optical capture device 708.Logic 726 causes the bit-mapped image to be stored in memory 730 with anassociated time-stamp read from the clock unit 728. Logic 726 may alsoperform optical character recognition (OCR) or other post-scanprocessing on the bit-mapped image to convert it to text. Logic 726 mayoptionally extract a signature from the image, for example by performinga convolution-like process to locate repeating occurrences ofcharacters, symbols or objects, and determine the distance or number ofother characters, symbols, or objects between these repeated elements.The reader may then upload the bit-mapped image (or text or othersignature, if post-scan processing has been performed by logic 726) toan associated computer via network interface 716.

As an example of another use of mobile device 702, a reader may capturesome text from an article as an audio file by using microphone 710 as anacoustic capture port. Logic 726 causes audio file to be stored inmemory 730. Logic 726 may also perform voice recognition or otherpost-scan processing on the audio file to convert it to text. As above,the reader may then upload the audio file (or text produced by post-scanprocessing performed by logic 726) to an associated computer via networkinterface 716.

A directional sensor 741 may also be incorporated into the mobile device702. The directional device may be a compass and be based upon amagnetic reading, or based upon network settings.

In the following sections, detailed descriptions of examples and methodsof the invention will be given. The description of both preferred andalternative examples though through are exemplary only, and it isunderstood that to those skilled in the art that variations,modifications and alterations may be apparent. It is therefore to beunderstood that the examples do not limit the broadness of the aspectsof the underlying invention as defined by the claims.

Referring now to FIG. 8, exemplary steps that may be performed in someaspects of the present invention are illustrated. At step 801, aprocessor may generate an AVM model of a processing facility. The AVMmodel may be based upon a physical layout of the processing facility andinclude a layout of each item of machinery, equipment as well asfacility features. At step 802, the AVM may receive data indicative ofone or more performance metrics. Data may include data generated via asensor and/or input by a user. In some examples, data may includeperformance metrics, utility cost, maintenance cost and replacementcost.

At step 803, a data connection between a deployed facility and an AVMmay be automated to generate and transmit data to the model on anautomated basis without human intervention or artificial delay. All orsome data may be stored in a storage. At step 804, the AVM may accessreceived and/or historical data from the same or other AVM models. Atstep 805. Artificial Intelligence routines or other logic may integraterelevant indices, including one or more of: geographic location, labororganization, market conditions, labor costs, physical conditions,property status or data descriptive of other variables.

At step 806, an AVM may generate a value for build and deployment cost,and at step 807 the AVM may include utility and consumables cost. Atstep 808 an AVM may generate one or more of: predicted and actualquantifications from the structure; energy consumption and processthroughput.

Referring now to FIG. 9A, an exemplary perspective graph 900 comprisingthree separate perspective points 925, 945, 965 is illustrated. In someaspects, as illustrated in FIG. 9B, a wearable display 905 may beconfigured to detect eye movement of the wearer 915, which may becalibrated. For example, such as illustrated in FIG. 9B, a neutral,forward-looking eye position 920 may be established as the center pointof the axes 910 (0, 0), which may establish a view along the positivez-axis. As a further illustrative example in FIG. 9C, once calibrated, ashift in eye position 940 to look up and left may change a view from thevantage point and be transmitted to the AVM to access another portion ofthe AVM. As an illustrative example, as shown in FIG. 9D, a user maylook right, and the eye position 960 may shift along the positivex-axis.

In some aspects, the wearable display 905 may comprise a set of gogglesor glasses, wherein the goggles or glasses may comprise one or morelenses. For example, a single wrapped lens may allow a user toexperience panoramic views. Alternately, dual lenses may providedifferent image data, wherein the combined images may allow the user tohave stereoscopic perception of the performance event. In still furtherembodiments, the wearable display 905 may comprise a helmet, which mayallow for more detailed immersion. For example, a helmet may allow fortemperature control, audio isolation, broader perspectives, orcombinations thereof.

Referring now to FIGS. 10A-10C, exemplary horizontal changes in viewingareas are illustrated. In some embodiments, the wearable display maycomprise an accelerometer configured to detect head movement. Similarlyto the eye position detection, the accelerometer may be calibrated tothe natural head movements of a user 1000. In some embodiments, thecalibration may allow the user to tailor the range to the desiredviewing area. For example, a user may be able to move their head 110°comfortably, and the calibration may allow the user to view the entire180° relative the natural 110° movement.

As illustrated in FIG. 10A, a neutral head position 1020 of the wearabledisplay may allow the user 1000 to view a forward-looking perspective1025. As illustrated in FIG. 10B, a right head position 1040 of thewearable display may allow the user 1000 to view a rightward-lookingperspective 1045. As illustrated in FIG. 10C, a left head position 1060of the wearable display may allow the user 1000 to view aleftward-looking perspective 1065.

Referring now to FIGS. 11A-11C, exemplary vertical changes in viewingareas are illustrated. Similarly to FIGS. 10A-10C, in some embodiments,the wearable display may be configured to detect vertical motions. Insome aspects, a user may look up to shift the viewing area to a range inthe positive y axis grids, and user may look down to shift the viewingarea to a range in the negative y axis grids. In some embodiments, thewearable display may be configured to detect both horizontal andvertical head motion, wherein the user may be able to have almost a 270°viewing range.

As illustrated in FIG. 11A, a neutral head position 1120 of the wearabledisplay may allow the user 1100 to view a forward-looking perspective1125. As illustrated in FIG. 11B, an up head position 1140 of thewearable display may allow the user 1000 to view an upward-lookingperspective 1145. As illustrated in FIG. 11C, a down head position 1160of the wearable display may allow the user 1100 to view adownward-looking perspective 1165.

In still further embodiments, the wearable display may be able to detect360° of horizontal movement, wherein the user may completely turn aroundand change the neutral viewing range by 180°. In some aspects, thewearable display may be configured to detect whether the user may besitting or standing, which may shift the perspective and viewing area.In some implementations, a user may be allowed to activate or deactivatethe motion detection levels, based on preference and need. For example,a user may want to shift between sitting and standing throughout theexperience without a shift in perspective. In some implementations, thewearable display may further comprise speakers, wherein audio data maybe directed to the user.

In some embodiments, the wearable display may allow for immersion levelcontrol, wherein a user may adjust the level of light and transparencyof the wearable display and/or frames. In some aspects, the lenses ofthe wearable display may comprise an electrically active layer, whereinthe level of energy may control the opacity. For example, theelectrically active layer may comprise liquid crystal, wherein theenergy level may control the alignment of the liquid crystal. Where auser may prefer a fully immersive viewing experience, the lenses may beblacked out, wherein the user may see the video with minimal externalvisibility. Where a user may still prefer to have awareness orinteractions beyond the video, the lenses and/or frames may allow forsome light to penetrate or may allow for some transparency of the video.

Additional examples may include Sensor arrays, audio capture arrays andcamera arrays with multiple data collection angles that may be complete360 degree camera arrays or directional arrays, for example, in someexamples, a Sensor array (including image capture Sensors) may includeat least 120 degrees of data capture, additional examples include aSensor array with at least 180 degrees of image capture; and still otherexamples include a Sensor array with at least 270 degrees of imagecapture. In various examples, data capture may include Sensors arrangedto capture image data in directions that are planar or oblique inrelation to one another.

Referring now to FIG. 12, methods and devices for determining adirection that may be referenced for one or both of data capture and AVMpresentation of a particular portion of the virtual representation ofthe modeled structure. A User 1200 may position a Smart Device 1205 in afirst position 1201 proximate to a portion of a structure for which arepresentation in the AVM the User 1200 wishes to retrieve and display.The first position 1201 of the Smart Device 1205 may be determined (asdiscussed herein via GPS and/or triangulation) and recorded. The User1200 may then relocate the Smart Device 1205 to a second position 1202in a general direction of the portion of a structure (illustrated as theZ direction) for which a representation in the AVM the User 1200 wishesto retrieve and display. In this manner, the AVM system (not shown inFIG. 12) and/or the Smart Device 1205 may generate a vector towards theportion of a structure for which a representation in the AVM the User1200 wishes to retrieve and display.

In some embodiments, the vector may have a length determined by the AVMthat is based upon a length of a next Feature in the AVM located in thedirection of the generated vector. The vector will represent a distance1203 from the second position 1202 to an item 1225 along the Z axisdefined by a line between the first position 1201 and the secondposition 1202.

As illustrated, the change in the Z direction is associated with a zerochange in the X and Y directions. The process may also include a secondposition 1205 that has a value other than zero in the X and/or Ydirections.

In other embodiments, a User 1200 may deploy a laser, accelerometer,sound generator or other device to determine a distance from the SmartDevice 1205 to the feature, such as a piece of equipment. Such uniquemethods of determining a location and direction of data capture may beutilized to gather data during construction of modeled buildings orother structures and during Deployment of the structures during theOperational Stage. An additional non-limiting example may includedirection based identification; with a fixed location, or in tandem witha location means, a device may have capabilities to deduce orientationbased information of the device. This orientation information may beused to deduce a direction that the device is pointing in. Thisdirection based information may be used to indicate that the device ispointing to a specific piece of equipment 1225 that may be identified inthe AVM.

In still other embodiments, a device with a controller and anaccelerometer, such as mobile Smart Device 1205, may include a userdisplay that allows a direction to be indicated by movement of thedevice from a determined location acting as a base position towards anAs Built feature in an extended position. In some implementations, theSmart Device determines a first position 1201 based upon triangulationwith the reference points. The process of determination of a positionbased upon triangulation with the reference points may be accomplished,for example via executable software interacting with the controller inthe Smart Device, such as, for example by running an app on the SmartDevices 1205.

In combination with, or in place of directional movement of a SmartDevice 1205 in order to quantify a direction of interest to a user, someembodiments may include an electronic and/or magnetic directionalindicator that may be aligned by a user in a direction of interest.Alignment may include, for example, pointing a specified side of adevice, or pointing an arrow or other symbol displayed upon a userinterface on the device towards a direction of interest.

In a similar fashion, triangulation may be utilized to determine arelative elevation of the Smart Device as compared to a referenceelevation of the reference points.

It should be noted that although a Smart Device is generally operated bya human user, some embodiments of the present invention include acontroller, accelerometer, and data storage medium, Image CaptureDevice, such as a Charge Coupled Device (“CCD”) capture device and/or aninfrared capture device being available in a handheld or unmannedvehicle.

An unmanned vehicle may include for example, an unmanned aerial vehicle(“UAV”) or ground level unit, such as a unit with wheels or tracks formobility and a radio control unit for communication.

In some embodiments, multiple unmanned vehicles may capture data in asynchronized fashion to add depth to the image capture and/or a threedimensional and 4 dimensional (over time) aspect to the captured data.In some implementations, UAV position will be contained within aperimeter and the perimeter will have multiple reference points to helpeach UAV (or other unmanned vehicle) determine a position in relation tostatic features of a building within which it is operating and also inrelation to other unmanned vehicles. Still other aspects includeunmanned vehicles that may not only capture data but also function toperform a task, such as paint a wall, drill a hole, cut along a definedpath, or other function. As stated throughout this disclosure, thecaptured data may be incorporated into an AVM.

In still other embodiments, captured data may be compared to a libraryof stored data using recognition software to ascertain and/or affirm aspecific location, elevation and direction of an image capture locationand proper alignment with the virtual model. Still other aspects mayinclude the use of a compass incorporated into a Smart Device.

By way of non-limiting example, functions of the methods and apparatuspresented herein may include one or more of the following factors thatmay be modeled and/or tracked over a defined period of time, such as,for example, an expected life of a build (such as, 10 years or 20years).

Particular embodiments of the subject matter have been described. Otherembodiments are within the scope of the following claims. In some cases,the actions recited in the claims can be performed in a different orderand still achieve desirable results. In addition, the processes depictedin the accompanying figures do not necessarily require the particularorder show, or sequential order, to achieve desirable results. Incertain implementations, multitasking and parallel processing may beadvantageous. Nevertheless, it will be understood that variousmodifications may be made without departing from the spirit and scope ofthe claimed invention.

What is claimed is:
 1. Apparatus for augmenting a virtual model of a physical structure, the apparatus comprising: a smart device comprising a processor in logical communication with a communications network, said smart device additionally comprising a storage device, the storage device storing executable software that is executable upon demand and operative with the processor to: communicate with a server to access a virtual model file of a physical structure based upon a geolocation of the physical structure, the virtual model comprising digital representation of structural aspects of the physical structure, utility support within the physical structure and technical specifications associated with the physical structure; with the smart device at a first position, reference multiple positioning reference devices within the physical structure; determining a distance to at least three of the positioning reference devices with the smart device at the first position; generate a first X, Y and Z value of the smart device within the physical structure based upon the determined distance of the at least three positioning reference devices to the smart device at the first position and a triangulation calculation; specify a direction of interest via the User extending the smart device from the first position comprising a base position to a second position comprising an extended position; determine a distance to at least three of the positioning reference devices from the smart device at the second position; calculate a second X, Y and Z value of the smart device at the second position based upon a determined distance of the at least three positioning reference devices to the smart device at the second position and a triangulation calculation; generate a direction of interest based upon a directional vector originating at the first position X, Y and Z value and continuing through the second position X, Y and Z value; display a user interface on the smart device, the user interface comprising a symbol indicating the direction of interest; indicate a Z distance of interest from a data capture position comprising one of: a) the first position X, Y and Z value, and b) the second X, Y and Z value of the smart device; perform a data retrieval routine from the data capture position, said data retrieval routine accessing a digital record of a feature of the physical structure included in the virtual model, based upon the distance of interest and the direction of interest from the data capture position of the smart device; and display a virtual representation of the digital record with the user interface, the virtual representation comprising the feature of the physical structure within the virtual model at a virtual location based upon the data capture position and the distance of interest and the direction of interest.
 2. The apparatus of claim 1 wherein the software is additionally operative to transmit the data capture position of the smart device to the virtual model.
 3. The apparatus of claim 2 wherein at least one of the multiple positioning reference devices comprises a first radio frequency transmitter fixedly attached to a first reference position within the physical structure and the software is additionally operative to: a) transmit to the virtual model an identification of the first radio frequency transmitter; and b) transmit an instruction to store an identification of the first radio frequency transmitter in relation to the performance of the data retrieval routine.
 4. The apparatus of claim 3 wherein at least one other position referencing device comprises a second radio frequency transmitter fixedly attached to a second reference position and associated with an identification of the second radio frequency transmitter, and the software is additionally operative with the processor to: a) transmit to the virtual model the identification of the second radio frequency transmitter; and b) transmit an instruction to store the identification of the second radio frequency transmitter in relation to the performance of the data retrieval routine.
 5. The apparatus of claim 3 wherein at least one position referencing device comprises a visual indicator, and the software is additionally operative with the processor to: a) transmit to the virtual model an identification of the visual indicator; and b) transmitting an instruction to store the identification of the visual indicator in relation to the performance of the data retrieval routine.
 6. The apparatus of claim 3 wherein the software is additionally operative with the processor to display image data within a field of view of the smart device at the distance of interest and the direction of interest from the position of the smart device.
 7. The apparatus of claim 6 wherein the software is additionally operative with the processor to display captured image included in the virtual model at a virtual position congruent with the distance of interest and the direction of interest from the X, Y and Z position.
 8. The apparatus of claim 7 wherein the image display is accomplished with a human readable display included in the smart device.
 9. The apparatus of claim 3 wherein the data retrieval routine comprises obtaining captured data from a sensor reading based upon a physical condition of the physical structure measured at the distance of interest and the direction of interest from the X, Y and Z position of the smart device and the elevation of the smart device.
 10. The apparatus of claim 9 wherein the physical condition comprises an indication of an amount of vibration.
 11. The apparatus of claim 9 wherein the physical condition comprises one or both of a temperature and a humidity present in the physical structure at the distance of interest and the direction of interest from the X, Y and Z position of the smart device.
 12. The apparatus of claim 9 wherein the software is additionally operative with the processor to: associate the distance of interest and the direction of interest from the X, Y and Z position of the smart device within a physical embodiment of the physical structure with a nomenclature; and display captured image data within the virtual model, based upon a user entry of a query including the nomenclature.
 13. The apparatus of claim 12 wherein the software is additionally operative with the processor to: a) provide an index of nomenclatures; associating each nomenclature with a virtual position and direction within the virtual model; and b) display captured data based upon a user selection of nomenclature.
 14. The apparatus of claim 13 wherein the software is additionally operative with the processor to display one or more utility support apparatus in the virtual model based upon a physical location of a user interface display device used to display the virtual model.
 15. The apparatus of claim 13 wherein the data retrieval routine comprises retrieving a state of one or more utility support apparatus comprising one or both of: plumbing and electrical components.
 16. The apparatus of claim 15 wherein the wherein the software is additionally operative with the processor to update technical specifications regarding one or more utility support apparatus based upon a user entry into the user interface.
 17. The apparatus of claim 16 wherein the updated technical specifications include a change in a type of one or both of the plumbing components and electrical components.
 18. The apparatus of claim 16 wherein the updated technical specifications include a change in a location of one or both of the plumbing components and electrical components.
 19. The apparatus of claim 16 wherein the updated technical specifications include a volume of production of a machine located at the distance of interest and the direction of interest from the X, Y and Z position of the smart device within the physical embodiment of the physical structure.
 20. The apparatus of claim 16 wherein the updated technical specifications include a quality of production of a machine located at the distance of interest and the direction of interest from the X, Y and Z position of the smart device of the smart device within the physical embodiment of the physical structure. 